Adaptive facial expression calibration

Technologies for generating an avatar with a facial expression corresponding to a facial expression of a user include capturing a reference user image of the user on a computing device when the user is expressing a reference facial expression for registration. The computing device generates reference facial measurement data based on the captured reference user image and compares the reference facial measurement data with facial measurement data of a corresponding reference expression of the avatar to generate facial comparison data. After a user has been registered, the computing device captures a real-time facial expression of the user and generates real-time facial measurement data based on the captured real-time image. The computing device applies the facial comparison data to the real-time facial measurement data to generate modified expression data, which is used to generate an avatar with a facial expression corresponding with the facial expression of the user.

BACKGROUND

Avatars are becoming an integral part of how we communicate with one another in a digital world. The potential uses for avatars range from increased immersion of a player into a role-playing video game environment to protecting a person's privacy when engaged in remote video chat. In any case, advances in technology and computing algorithms enhance the realism of communication via an avatar. Avatars in video games allow user's to control the facial gestures of mystical creatures and characters with their own facial expressions. Also, in the context of remote video chat, advanced avatars are nearly indistinguishable from actual real-time video of the user.

Although advances in technology typically unleash endless possibilities and near-reality experiences, it comes at a cost. The computational resources typically required for video editing and rendering are considerable, particularly when it comes to rendering life-like video and motion. For example, many applications require devices to have significant graphical processing capabilities and dedicated memory. As such, low-power mobile computing devices are typically unable to implement the certain technologies as effectively as desktop computers and servers. Instead, those devices often must resort to last generation's applications.

DETAILED DESCRIPTION OF THE DRAWINGS

Referring now toFIG. 1, a system100for generating an avatar with a facial expression corresponding to a facial expression of a user includes a computing device102, a network104, and an avatar database106. In use, as discussed in more detail below, the computing device102may retrieve data from the avatar database106and generate an avatar with a facial expression corresponding to a facial expression of a user (e.g., mimicking the user). Although only one computing device102, one network104, and one avatar database106are illustratively shown inFIG. 1, the system100may include any number of computing devices102, networks105, and avatar databases106. For example, the system100may include several computing devices102utilizing the same avatar database106to generate avatars.

The computing device102may be embodied as any type of computing device capable of establishing a communication link with avatar database106and performing the functions described herein. For example, the computing device102may be embodied as a cellular phone, smartphone, tablet computer, laptop computer, personal digital assistant, mobile Internet device, desktop computer, server, and/or any other computing/communication device. As shown inFIG. 1, the illustrative computing device102includes a processor120, an input/output (“I/O”) subsystem122, a memory124, a communication circuitry126, a data storage128, one or more peripheral devices130, one or more cameras132, and a display134. Of course, the computing device102may include other or additional components, such as those commonly found in a typical computing device (e.g., various input/output devices), in other embodiments. Additionally, in some embodiments, one or more of the illustrative components may be incorporated in, or otherwise from a portion of, another component. For example, the memory124, or portions thereof, may be incorporated in the processor120in some embodiments.

The processor120may be embodied as any type of processor capable of performing the functions described herein. For example, the processor may be embodied as a single or multi-core processor(s), digital signal processor, microcontroller, or other processor or processing/controlling circuit. Similarly, the memory124may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. In operation, the memory124may store various data and software used during operation of the computing device102such as operating systems, applications, programs, libraries, and drivers. The memory124is communicatively coupled to the processor120via the I/O subsystem122, which may be embodied as circuitry and/or components to facilitate input/output operations with the processor120, the memory124, and other components of the computing device102. For example, the I/O subsystem122may be embodied as, or otherwise include, memory controller hubs, input/output control hubs, firmware devices, communication links (i.e., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.) and/or other components and subsystems to facilitate the input/output operations. In some embodiments, the I/O subsystem122may form a portion of a system-on-a-chip (SoC) and be incorporated, along with the processor120, the memory124, and other components of the computing device102, on a single integrated circuit chip.

The communication circuitry126of the computing device102may be embodied as any communication circuit, device, or collection thereof, capable of enabling communications between the computing device102and other remote devices (e.g., the avatar database106) over the network104. The communication circuitry126may be configured to use any one or more communication technology (e.g., wireless or wired communications) and associated protocols to effect such communication.

The data storage128may be embodied as any type of device or devices configured for short-term or long-term storage of data such as, for example, memory devices and circuits, memory cards, hard disk drives, solid-state drives, or other data storage devices. In the illustrative embodiment ofFIG. 1, the data storage128includes facial data150and facial comparison data152. The facial data150includes facial geometry data and/or facial measurement data, which are discussed in detail below. The peripheral devices130of the computing device102may include any number of additional peripheral or interface devices. The particular devices included in the peripheral devices130may depend on, for example, the type and/or intended use of the computing device102.

The camera132may be embodied as any peripheral or integrated device suitable for capturing images, such as a still camera, a video camera, a webcam, or the like. The camera132may be used to capture images of the user's various facial expressions (e.g., a reference facial expression, a neutral facial expression, or real-time facial expression). The display134of the computing device102may be embodied as any one or more display screens on which information may be displayed to a viewer of the computing device102. The display may be embodied as, or otherwise use, any suitable display technology including, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, a cathode ray tube (CRT) display, a plasma display, and/or other display technology. Although only a single display134is illustrated inFIG. 1, it should be appreciated that the computing device102may include multiple displays or display screens on which the same or different content may be displayed contemporaneously or sequentially with each other.

The network104may be embodied as any number of various wired and/or wireless telecommunication networks. As such, the network104may include one or more networks, routers, switches, computers, and/or other intervening devices. For example, the network104may be embodied as or otherwise include one or more cellular networks, telephone networks, local or wide area networks, publicly available global networks (e.g., the Internet), or any combination thereof.

The avatar database106may be embodied as any electronic arrangement or structure suitable for storing data directed to avatars. In one embodiment, the avatar database106stores images of avatars with various facial expressions. Additionally or alternatively, the avatar database106may store facial geometry data and/or facial measurement data of avatars with different facial expressions (e.g., reference facial expression or neutral facial expression). In some embodiments, the avatar database106is stored on a remote computing device embodied as an enterprise-level server computer, a desktop computer, a laptop computer, a tablet computer, a cellular phone, smartphone, personal digital assistant, mobile Internet device, and/or any other computing/communication device. Further, in one embodiment, access to the avatar database106is managed by a web service server, which the computing device102may communicate with over the network104. Of course, in some embodiments, the avatar database106or data therefrom may be stored on the computing device102.

Referring now toFIG. 2, in use, the computing device102of the system100establishes an environment200for generating an avatar with a facial expression corresponding to a facial expression of a user. The environment200in the illustrative embodiment includes a expression generation module202, a rendering module212, and a communication module214. Additionally, the expression generation module202further includes a facial measurement module204, a facial comparison module206, an avatar adaptation module208, and a transformation module210. Each of the expression generation module202, the facial measurement module204, the facial comparison module206, the avatar adaptation module208, the transformation module210, the rendering module212, and the communication module214may be embodied as hardware, software, firmware, or a combination thereof.

As discussed in more detail below, the expression generation module202receives images of a user and avatar data (e.g., through the communication module214or from the memory124, the data storage128, or the camera132) and generates modified expression data corresponding to a modified facial expression of the avatar as a function of the user images and the avatar data. The modified expression data may then be used to render the avatar having a facial expression corresponding to a facial expression of the user (e.g., the user's current facial expression).

In doing so, the facial measurement module204performs feature detection on a facial image of the user or avatar and generates facial measurement data based on the image. The facial measurement data defines measurements of various facial features of user. For example, in one embodiment, the facial measurement module204extracts various features from the image (i.e., facial geometry data). In doing so, the facial measurement module204may employ any suitable feature extraction algorithm. For example, the feature extraction algorithm may identify facial landmarks, edges, and/or corners. In one embodiment, the facial measurement module204generates a polygon mesh, contour map, or wire frame model corresponding with the extracted facial geometry data. In another embodiment, the facial measurement module204simply stores the facial geometry data (e.g., the coordinates of the vertices in the polygon mesh or the extracted feature landmark points). It should be appreciated that the facial geometry data may be stored in any suitable data structure (e.g., as a vector, matrix, array, or n-tuple).

The facial measurement module204generates the facial measurement data based on the extracted facial geometry data. Specifically, various facial measurements may be predefined by the system100. For example, the facial measurements may include the distance between the centers of the eyes and the nose tip, the distance between the upper and lower lips, the distance between the nose wing and the nose tip, the distance between a portion of the eyebrow and a portion of the nose, the distance between the eyebrow and the eye, the distance between the eyebrow tips, the distance between the lower lip and the nose tip, the distance between the upper and lower eyelids, the distance between the side corners of the mouth, the distance between one part of an eyebrow and another part of the same eyebrow, the distance between the upper lip and the nose tip, and/or the distance between the lower lip and the nose tip. Regardless of the particular facial measurements utilized by the system100, the facial measurement data may be stored or represented in any suitable way (e.g., as a vector, matrix, array, or n-tuple). For example, the facial measurement data is organized or expressed as a shape vector such that each dimension of the shape vector corresponds with a different facial measurement. It should be appreciated that the particular facial measurements and number of facial measurements chosen in a particular embodiment may be carefully selected to accurately represent or reflect most facial expressions. Additionally, facial geometry data and facial measurement data may be generated for a facial image of a person or an avatar. In some embodiments, avatars may be mythical or other non-humanoid characters. As such, data or information may be provided regarding an avatar to indicate the locations of the features of the avatar if the features would not be readily identified via the implemented feature extraction algorithm. For example, facial geometry data and/or facial measurement data may be provided by the avatar database106for one or more facial expressions of an avatar (e.g., a reference facial expression).

In some embodiments, the images are required to be in a particular orientation (e.g., front perspective), size, shape, etc. in order to extract features and/or generate accurate facial measurement data. In those embodiments, the transformation module210works in tandem with the facial measurement module204. That is, the transformation module210may transform an image of the user and/or avatar to a particular perspective (e.g., via a similarity transform) in order to remove translation, rotation, and scaling factors. In some embodiments, the transformation module210ensures that the features are extracted and/or the measurements are made within the same plane and from the same depth (i.e., zoomed in by the same amount). Additionally, the transformation module210may transform an image of the user and/or avatar to standardize the images. In other words, the transformation module210may, in some embodiments, standardize or normalize images such that they have the same level of granularity or precision.

The facial comparison module206compares the facial measurement data of two parties (e.g., the user, actor, or avatar) to generate facial comparison data. For example, the facial comparison module206may compare the facial measurement data of a user's reference facial expression with the facial measurement data of an avatar's reference facial expression. Alternatively, the facial comparison module206may compare the facial measurement data of a user's reference facial expression with the facial measurement data of an actor's reference facial expression as discussed below in reference toFIG. 6. As discussed above, in some embodiments, the facial measurement data may be represented as a shape vector. In such an embodiment, the facial comparison module206may generate the facial comparison data by subtracting the facial measurement data of the user's reference facial expression from the facial measurement data of the avatar's facial expression data.

For example, suppose the facial measurement data of the user's reference facial expression is represented by the vector, SUSER, and the facial measurement data of the avatar's reference facial expression is represented by the vector, SAVATAR. Then, the facial comparison data may be represented by ΔS=SAVATAR−SUSER, the vector difference of SAVATARand SUSER. Of course, the facial comparison module206may compare facial measurement data of various parties using other suitable image processing techniques and algorithms. In some embodiments, facial measurement data of two images are compared only when the facial measurement data of each of those images is corresponding to the same reference facial expression (e.g., a neutral facial expression or a smiling facial expression). Further, in another embodiment, the facial comparison module206may directly compare the facial geometry data of one image to the facial geometry data of another image.

The avatar adaptation module208generates modified expression data of the avatar by applying the facial comparison data to real-time facial measurement data of the user. As discussed above, the modified expression data may then be used to render (e.g., using the rendering module212) the avatar having a facial expression corresponding to a facial expression of the user (e.g., the user's current facial expression). In some embodiments, the modified expression data represents facial measurement data for the modified or adapted avatar (i.e., the avatar with the desired modified facial expression), which may be used to generate the desired avatar. In other embodiments, the modified expression data may be other data (e.g., facial geometry data) used to generate the desired avatar. Additionally, the avatar adaptation module208may generate the modified expression data or otherwise generate the data for rendering the avatar using any suitable means.

In some embodiments, the avatar adaptation module208generates the modified expression data by adding the facial comparison data to real-time facial measurement data of the user. For example, return to the example discussed above in which shape vectors are used to represent the facial measurement data. In such an embodiment, the modified expression data for the avatar, S′AVATAR, may be represented by S′AVATAR=S′USER+ΔS, the vector sum of the real-time facial measurement data of the user and the facial comparison data. As discussed above, in some embodiments, an actor's facial measurement data may be used rather than an avatar's facial measurement data.

The rendering module212generates the desired avatar having a facial expression corresponding to the current facial expression of the user based on the modified expression data generated by the avatar adaptation module208. That is, in some embodiments, the rendering module212generates a visual depiction of the modified avatar as a function of the modified expression data. The modified avatar may be displayed on the display134of the computing device102. Of course, if the modified avatar is to be displayed on a remote computing device, the modified avatar may be transmitted to the remote computing device using the communication module214.

The communication module214handles the communication between the computing device102and remote devices (e.g., the avatar database106) through the network104. As such, the communication module214may receive data (e.g., images of the avatar(s) or actors, facial geometry data, and/or facial measurement data) from the avatar database106or other remote device and provide the data to the expression generation module202. Additionally, in some embodiments, the communication module214may transmit the modified expression data to a remote computing device for rendering in addition or in the alternative to rendering the avatar on the computing device102using the rendering module212. It should be appreciated that the expression generation module202and rendering module212may store the data at any one or more stage depending on the embodiment. For example, the images, the facial measurement data, facial geometry data, facial comparison data, modified expression data, and/or rendered avatars may be stored on the computing device102for later retrieval. Relatedly, in some embodiments, the facial comparison data is only generated once (i.e., during “registration”) on the computing device102for any given user-avatar or user-actor pair and that information is stored on the computing device102for later use.

Referring now toFIG. 3, in use, the computing device102of the system100may execute a method300for generating an avatar with a facial expression corresponding to a facial expression of a user. For example, the user may register her face with the computing device102for a particular avatar. The computing device102may then capture (e.g., continuously or periodically) the user's real-time facial expression and generate an avatar with a facial expression corresponding to the user's current facial expression (e.g., a matching facial expression). The illustrative method300begins with block302in which the computing device102determines whether the user is registered for a particular avatar. As such, in some embodiments, a particular user may register for multiple avatars. Accordingly, the avatar may choose which avatar to use in some embodiments. If the user has not registered for the avatar, the computing device102instructs the user to present a reference facial expression to the camera132(e.g., a neutral facial expression) in block304.

In block306, the computing device102captures one or more images of the user's reference facial expression using the camera132. In block308, the computing device102generates and stores facial comparison data corresponding with the reference facial expression (seeFIG. 4). In other words, the computing device102registers the user for the particular avatar. If the computing device102determines in block302that the user has been registered for the avatar, the computing device102retrieves the user's facial comparison data for the avatar in block310.

In block312, the computing device102generates real-time facial geometry data of the user's current expression. In doing so, the computing device102captures a current real-time image of the user and extracts the facial geometry data of the real-time image as described above. In block314, the computing device102generates real-time facial measurement data of the user's current expression based on the real-time facial geometry data. The computing device102generates modified expression data for the avatar in block316. As discussed above, in some embodiments, the modified expression data comprises the facial measurement data for the modified expression of the avatar (i.e., the expression corresponding with the real-time facial expression of the user). In generating the modified expression data, the computing device102applies the facial comparison data to the real-time facial measurement data of the user's current expression in block318.

In block320, the computing device102renders and/or transmits the modified avatar using the modified expression data. As discussed above, in some embodiments, the avatar is rendered for display on the display134of the computing device102, whereas in other embodiments, the avatar may be transmitted over the network104to another computing device102for display on the other computing device102. Of course, in one embodiment in which the avatar is transmitted for display on a remote computing device, the computing device102transmits the modified expression data to the remote computing device rather than a rendered avatar.

Referring now toFIG. 4, in use, the computing device102of the system100may execute a method400for generating and storing facial comparison data of a reference expression of a user and avatar. The method400begins with block402in which the computing device102generates facial geometry data of the user's reference expression. The reference expression may be, for example, a neutral facial expression or other distinguishable facial expression of the user (e.g., a smiling facial expression or frowning smiling expression). Further, as discussed above, the facial geometry data may be generated using a suitable feature extraction algorithm to identify geometric and/or topological markers on the image of the user's reference expression.

In block404, the computing device102generates facial measurement data of the user's reference facial expression based on the generated facial geometry data. In doing so, in some embodiments, the computing device102may generate a shape vector in block406. As discussed above, in other embodiments, the facial measurement data may be expressed in any other suitable form. In block408, the computing device102accesses the facial measurement data of the avatar (i.e., the chosen avatar). The facial measurement data corresponds to an image of the avatar's reference expression. In other words, if the user's reference facial expression is a neutral facial expression, then the avatar's reference expression should also be a neutral facial expression. It should be appreciated that the computing device102may access the measurement data of the avatar's reference expression in various ways. In one embodiment, in block410, the computing device102simply retrieves the facial measurement data associated with the avatar's reference expression (e.g., from the avatar database106over the network104or from the memory124or the data storage128of the computing device102).

In another embodiment, the computing device102receives an image of the avatar with a reference facial expression (i.e., from the avatar database106, the memory124, or the data storage128) and generates the facial measurement data of the avatar's reference facial expression similar to the facial measurement data of the user's reference facial expression. Specifically, in block412, the computing device102receives the reference image of the avatar. In block414, the computing device102generates facial geometry data of the avatar's reference expression based on the avatar image. Using that facial geometry data, the computing device102generates the facial measurement data in block416.

As discussed above, in some embodiments, the computing device102must standardize or normalize the facial data (i.e., the facial geometry data and/or the facial measurement data) of the user and/or or the avatar in block418. Of course, in some embodiments, the standardization may occur coincident with or as part of the generation of the facial geometry data or the facial measurement data. For example, in one embodiment, the facial geometry data of the user's expression may indicate 17 points of interest (e.g., feature detection points or feature markers) and the facial geometry data of the avatar's expression may indicate 24 points of interest. The computing device102may standardize the facial geometry data of the two expressions such that each has the same number of points of interest (e.g., 17, 20, or 24). In doing so, the computing device102may utilize any suitable image-processing algorithm or process. For example, the computing device102may use interpolation (e.g., linear or polynomial interpolation), averaging, and/or best-fit algorithms.

In block420, the computing device102compares the facial measurement data of the user's reference facial expression to the facial measurement data of the avatar's reference facial expression to generate facial comparison data. As discussed above, the computing device102may do so by subtracting the facial measurement data of the user's reference facial expression from the facial measurement data of the avatar's reference facial expression. Further, the facial comparison data may be used to generate an avatar with a facial expression corresponding (e.g., mimicking) to the real-time facial expression of the user. As such, the computing device102stores the facial comparison data in block422.

Simplified data flow diagrams of at least two embodiments of methods for generating an avatar with a facial expression corresponding to a facial expression of a user are illustrated inFIGS. 5 and 6. Referring now toFIG. 5, which corresponds with the method300, the data flow500begins with block502in which the computing device102receives an image of the user's reference facial expression (e.g., using the camera132). The computing device102generates facial geometry data for the user's reference facial expression in block504, which is used to generate facial measurement data in block506. In block508, the computing device102receives an image of the selected avatar's reference facial expression (e.g., from the avatar database106) and generates the associated facial measurement data in block510(e.g., using generated facial geometry data for the avatar's reference facial expression). As discussed above, in some embodiments, the computing device102may not receive an image of the avatar but receive the facial measurement data or facial geometry data of the avatar's reference facial expression instead. In block512, the computing device102generates facial comparison data as a function of the user's facial measurement data and the avatar's facial measurement data. A real-time image of the user's current facial expression is captured in block514, which is used with the facial comparison data to generate the modified expression data for the modified avatar in block516. In block518, the desired avatar is rendered as a function of the modified expression data.

As discussed above, in some embodiments, an image of an actor may be used rather than an image of an avatar to generate an avatar with a facial expression corresponding to a facial expression of a user. For example, the avatar database106may include various images of a particular actor and associate each of those images with a corresponding image of the avatar. That is, an image of the actor smiling may correspond with an image of the avatar smiling, and an image of the actor with a neutral facial expression may correspond with an image of the avatar with a neutral facial expression. In such an embodiment, a method similar to the method300may be implemented in which the user's facial expressions are compared to the actor's facial expressions rather than the avatar's facial expressions.

Referring now toFIG. 6, the data flow600begins with block602in which the computing device102receives an image of the user's reference facial expression (e.g., using the camera132). The computing device102generates facial geometry data for the user's reference facial expression in block604, which is used to generate facial measurement data in block606. In block508, the computing device102receives an image of the actor's reference facial expression (e.g., from the avatar database106) and generates the associated facial measurement data in block610(e.g., using generated facial geometry data for the actor reference facial expression). Of course, in some embodiments, the computing device102may not receive an image of the actor but receive the facial measurement data or facial geometry data of the actor's reference facial expression instead. In block612, the computing device102generates facial comparison data as a function of the user's facial measurement data and the actor's facial measurement data. A real-time image of the user's current facial expression is captured in block614, which is used with the facial comparison data to generate the modified expression data for the modified actor in block616. It should be appreciated that only a finite number of images of the actor may be stored on the avatar database106. As such, in some embodiments, the modified expression data for the modified actor may not perfectly match the modified expression data (e.g., the facial measurement data or facial geometry data) for any of the stored actor images. Accordingly, the computing device102may quantize (also referred to as “discretize” or “round”) the modified expression data for the modified actor to match that of one of the existing actor images in the avatar database106in block618. Because the avatar database106maps each actor image to a corresponding avatar image, the computing device102is able to identify the desired avatar image and retrieve the avatar with the modified expression in block620.

EXAMPLES

Example 1 includes a computing device for generating an avatar with a facial expression corresponding to a facial expression of a user, the computing device comprising a camera to (i) capture a reference user image of a reference facial expression of the user and (ii) capture a real-time user image of a current facial expression of the user; an expression generation module to (i) generate reference facial measurement data, based on the reference user image, that defines measurements of facial features of the user, (ii) compare the reference facial measurement data with facial measurement data of a corresponding reference expression of the avatar to generate facial comparison data, (iii) generate real-time facial measurement data, based on the real-time user image, that defines measurements of facial features of the user, and (iv) apply the facial comparison data to the real-time facial measurement data to generate modified expression data; and a rendering module to generate the avatar having a facial expression corresponding to the current facial expression of the user as a function of the modified expression data.

Example 2 includes the subject matter of Example 1, and wherein the reference user image is an image of a neutral facial expression of the user.

Example 3 includes the subject matter of any of Examples 1 and 2, and wherein the expression generation module is to generate reference facial geometry data, based on the reference user image, that defines the facial features of the user; and generate the reference facial measurement data as a function of the reference facial geometry data.

Example 4 includes the subject matter of any of Examples 1-3, and wherein the reference facial measurement data comprises a shape vector.

Example 5 includes the subject matter of any of Examples 1-4, and wherein to generate the facial comparison data comprises to subtract the reference facial measurement data from the facial measurement data of the corresponding reference expression of the avatar.

Example 6 includes the subject matter of any of Examples 1-5, and wherein to apply the facial comparison data comprises to add the facial comparison data to the real-time facial measurement data to generate the modified expression data.

Example 7 includes the subject matter of any of Examples 1-6, wherein the reference facial measurement data comprises a first shape vector and the facial measurement data of the corresponding reference expression of the avatar comprises a second shape vector.

Example 8 includes the subject matter of any of Examples 1-7, and wherein the expression generation module is to (i) receive a reference avatar image of the reference expression of the avatar and (ii) generate the facial measurement data of the corresponding reference expression of the avatar based on the reference avatar image.

Example 9 includes the subject matter of any of Examples 1-8, and wherein the expression generation module is to generate facial geometry data of the reference expression of the avatar, based on the reference avatar image, that defines the facial features of the avatar; and generate the facial measurement data of the corresponding reference expression of the avatar as a function of the facial geometry data of the reference expression of the avatar.

Example 10 includes the subject matter of any of Examples 1-9, and wherein the expression generation module is to standardize the reference facial measurement data and the facial measurement data of the corresponding reference expression of the avatar to generate reference facial measurement data and facial measurement data of the corresponding reference expression of the avatar with a same number of measurements of the facial features.

Example 11 includes a method for generating an avatar with a facial expression corresponding to a facial expression of a user, the method comprising generating, on a computing device, reference facial measurement data, based on a reference user image of a reference facial expression of the user, that defines measurements of facial features of the user; generating, on the computing device, facial comparison data by comparing the reference facial measurement data to facial measurement data of a corresponding reference expression of the avatar; generating, on the computing device, real-time facial measurement data, based on a real-time user image of a current facial expression of the user, that defines measurements of facial features of the user; generating, on the computing device, modified expression data by applying the facial comparison data to the real-time facial measurement data; and rendering, on the computing device, the avatar having a facial expression corresponding to the current facial expression of the user as a function of the modified expression data.

Example 12 includes the subject matter of Example 11, and wherein generating the reference facial measurement data comprises generating reference facial measurement data based on a reference user image of a neutral facial expression of the user.

Example 13 includes the subject matter of any of Examples 11 and 12, and further including generating reference facial geometry data, based on the reference user image, that defines the facial features of the user, and wherein generating the reference facial measurement data comprises generating reference facial measurement data as a function of the reference facial geometry data.

Example 14 includes the subject matter of any of Examples 11-13, and wherein generating the reference facial measurement data comprises generating a shape vector.

Example 15 includes the subject matter of any of Examples 11-14, and wherein generating the facial comparison data comprises subtracting the reference facial measurement data from the facial measurement data of the corresponding reference expression of the avatar.

Example 16 includes the subject matter of any of Examples 11-15, and wherein generating the facial comparison data comprises subtracting a first shape vector from a second shape vector, the reference facial measurement data being the first shape vector and the facial measurement data of the corresponding reference expression of the avatar being the second shape vector.

Example 17 includes the subject matter of any of Examples 11-16, and wherein generating the modified expression data comprises adding the facial comparison data to the real-time facial measurement data.

Example 18 includes the subject matter of any of Examples 11-17, and wherein generating the modified expression data comprises adding a first shape vector to a second shape vector, the facial comparison data being the first shape vector and the real-time facial measurement data being the second shape vector.

Example 19 includes the subject matter of any of Examples 11-18, and further including receiving, with the computing device, a reference avatar image of the reference expression of the avatar; and generating, on the computing device, facial measurement data of the corresponding reference expression of the avatar based on the reference avatar image.

Example 20 includes the subject matter of any of Examples 11-19, and further including generating facial geometry data of the reference expression of the avatar, based on the reference avatar image, that defines the facial features of the avatar; and wherein generating the facial measurement data of the corresponding reference expression of the avatar comprises generating facial measurement data of the corresponding reference expression of the avatar as a function of the facial geometry data of the reference expression of the avatar.

Example 21 includes the subject matter of any of Examples 11-20, and further including standardizing, on the computing device, the reference facial measurement data and the facial measurement data of the corresponding reference expression of the avatar by generating reference facial measurement data and facial measurement data of the corresponding reference expression of the avatar with a same number of measurements of the facial features.

Example 22 includes the subject matter of any of Examples 11-21, and further including capturing, with a camera of the computing device, the reference user image and the real-time user image.

Example 24 includes one or more machine readable storage media comprising a plurality of instructions stored thereon that in response to being executed result in a computing device performing the method of any of Examples 11-22.

Example 25 includes a computing device for generating an avatar with a facial expression corresponding to a facial expression of a user, the computing device comprising means for generating reference facial measurement data, based on a reference user image of a reference facial expression of the user, that defines measurements of facial features of the user; means for generating facial comparison data by comparing the reference facial measurement data to facial measurement data of a corresponding reference expression of the avatar; means for generating real-time facial measurement data, based on a real-time user image of a current facial expression of the user, that defines measurements of facial features of the user; means for generating modified expression data by applying the facial comparison data to the real-time facial measurement data; and means for rendering the avatar having a facial expression corresponding to the current facial expression of the user as a function of the modified expression data.

Example 26 includes the subject matter of Example 25, and wherein the means for generating the reference facial measurement data comprises means for generating reference facial measurement data based on a reference user image of a neutral facial expression of the user.

Example 27 includes the subject matter of any of Examples 25 and 26, and further including means for generating reference facial geometry data, based on the reference user image, that defines the facial features of the user, and wherein the means for generating the reference facial measurement data comprises means for generating reference facial measurement data as a function of the reference facial geometry data.

Example 28 includes the subject matter of any of Examples 25-27, and wherein the means for generating the reference facial measurement data comprises means for generating a shape vector.

Example 29 includes the subject matter of any of Examples 25-28, and wherein the means for generating the facial comparison data comprises means for subtracting the reference facial measurement data from the facial measurement data of the corresponding reference expression of the avatar.

Example 30 includes the subject matter of any of Examples 25-29, and wherein the means for generating the modified expression data comprises means for adding the facial comparison data to the real-time facial measurement.

Example 31 includes the subject matter of any of Examples 25-30, and wherein the reference facial measurement data comprises a first shape vector and the facial measurement data of the corresponding reference expression of the avatar comprises a second shape vector.

Example 32 includes the subject matter of any of Examples 25-31, and further including means for receiving a reference avatar image of the reference expression of the avatar; and means for generating facial measurement data of the corresponding reference expression of the avatar based on the reference avatar image.

Example 33 includes the subject matter of any of Examples 25-32, and further including means for generating facial geometry data of the reference expression of the avatar, based on the reference avatar image, that defines the facial features of the avatar; and wherein the means for generating the facial measurement data of the corresponding reference expression of the avatar comprises means for generating facial measurement data of the corresponding reference expression of the avatar as a function of the facial geometry data of the reference expression of the avatar.

Example 34 includes the subject matter of any of Examples 25-33, and further including means for standardizing the reference facial measurement data and the facial measurement data of the corresponding reference expression of the avatar by generating reference facial measurement data and facial measurement data of the corresponding reference expression of the avatar with a same number of measurements of the facial features.

Example 35 includes the subject matter of any of Examples 25-34, and further including means for capturing the reference user image and the real-time user image.

Example 36 includes a computing device for generating a desired avatar with a facial expression corresponding to a facial expression of a user using a database including a plurality of facial expressions of an actor, the computing device comprising a camera to (i) capture a reference user image of a reference facial expression of the user and (ii) capture a real-time user image of a current facial expression of the user; an expression generation module to (i) generate reference facial measurement data, based on the reference user image, that defines measurements of facial features of the user, (ii) compare the reference facial measurement data with facial measurement data of a corresponding reference expression of the actor to generate facial comparison data, (iii) generate real-time facial measurement data, based on the real-time user image, that defines measurements of facial features of the user, (iv) apply the facial comparison data to the real-time facial measurement data to generate modified expression data, and (v) select a facial expression of the actor from the plurality of facial expressions corresponding with the modified expression data; and a rendering module to select an avatar from the database corresponding to the selected facial expression of the actor to generate the desired avatar, each of the plurality of facial expressions of the actor being mapped to an avatar having a corresponding facial expression.

Example 37 includes the subject matter of Example 36, and wherein the expression generation module is to quantize the modified expression data to correspond with a closest match of the plurality of facial expressions of the actor.

Example 38 includes the subject matter of any of Examples 36 and 37, and wherein the reference user image is an image of a neutral facial expression of the user.

Example 39 includes the subject matter of any of Examples 36-38, and wherein the reference facial measurement data comprises a shape vector.

Example 40 includes a method for generating a desired avatar with a facial expression corresponding to a facial expression of a user using a database including a plurality of facial expressions of an actor, the method comprising generating, on a computing device, reference facial measurement data, based on the reference user image, that defines measurements of facial features of the user; generating, on the computing device, facial comparison data by comparing the reference facial measurement data to facial measurement data of a corresponding reference expression of the actor; generating, on the computing device, real-time facial measurement data, based on the real-time user image, that defines measurements of facial features of the user; generating, on the computing device, modified expression data by applying the facial comparison data to the real-time facial measurement data; selecting, on the computing device, a facial expression of the actor from the plurality of facial expressions corresponding with the modified expression data; and generating, on the computing device, the desired avatar by selecting an avatar from the database corresponding to the selected facial expression of the actor, each of the plurality of facial expressions of the actor being mapped to an avatar having a corresponding facial expression.

Example 41 includes the subject matter of Example 40, and further including quantizing the modified expression data to correspond with a closest match of the plurality of facial expressions of the actor.

Example 42 includes the subject matter of any of Examples 40 and 41, and wherein generating the reference facial measurement data comprises generating reference facial measurement data based on a reference user image of a neutral facial expression of the user.

Example 43 includes the subject matter of any of Examples 40-42, and wherein generating the reference facial measurement data comprises generating a shape vector.

Example 45 includes one or more machine readable storage media comprising a plurality of instructions stored thereon that in response to being executed result in a computing device performing the method of any of Examples 40-43.

Example 46 includes a computing device for generating a desired avatar with a facial expression corresponding to a facial expression of a user using a database including a plurality of facial expressions of an actor, the computing device comprising means for generating reference facial measurement data, based on the reference user image, that defines measurements of facial features of the user; means for generating facial comparison data by comparing the reference facial measurement data to facial measurement data of a corresponding reference expression of the actor; means for generating real-time facial measurement data, based on the real-time user image, that defines measurements of facial features of the user; means for generating modified expression data by applying the facial comparison data to the real-time facial measurement data; means for selecting a facial expression of the actor from the plurality of facial expressions corresponding with the modified expression data; and means for generating the desired avatar by selecting an avatar from the database corresponding to the selected facial expression of the actor, each of the plurality of facial expressions of the actor being mapped to an avatar having a corresponding facial expression.

Example 47 includes the subject matter of Example 46, and further including means for quantizing the modified expression data to correspond with a closest match of the plurality of facial expressions of the actor.

Example 48 includes the subject matter of any of Examples 46 and 47, and wherein the means for generating the reference facial measurement data comprises means for generating reference facial measurement data based on a reference user image of a neutral facial expression of the user.

Example 49 includes the subject matter of any of Examples 46-48, and wherein the means for generating the reference facial measurement data comprises means for generating a shape vector.