Augmented reality fashion

Various approaches discussed herein enable augmenting an image of a person wearing an article of clothing to provide an augmented reality (i.e., virtual) article of clothing to a viewer in order to overlay, replace, and/or augment an actual article of clothing being worn by the user. The augmented reality article of clothing is positioned on the user's real-world piece of clothing according to various markings and/or data that allow the augmented reality article of clothing to appear as if it were being worn by the user, even during movement of the user.

BACKGROUND

Increases in computer processing power, along with advanced graphics processors and three-dimensional (3-D) modeling techniques, have caused virtual reality to become part of the mainstream. From video games, to driving simulations, to navigational overlays, virtual reality is used by millions of people every day. As people increasingly become comfortable with using virtual and/or augmented reality applications, such as with portable and/or wearable computing devices, people will want to use the latest technology in real-time to express themselves with various changes to their appearance in a virtual world.

Virtual reality devices, such as headsets or goggles, are rapidly developing to the point where these devices should soon be widely available for various consumer applications. For example, virtual reality headsets that display images of a virtual world have been demonstrated at various events and application developers are preparing for their upcoming release. While these approaches satisfy some needs, they do not provide a truly immersive environment.

DETAILED DESCRIPTION

Systems and methods in accordance with various embodiments of the present disclosure may overcome one or more of the aforementioned and other deficiencies experienced in conventional approaches for providing an augmented reality viewing experience where different users may view the same person, for example through a computing device or virtual reality device (e.g., headset, goggles, glasses, contact lens, etc.) and see that person wearing articles of clothing with different appearances depending on any number of criteria. In particular, various embodiments enable a user to wear an article of clothing (e.g., a shirt, etc.) that in various embodiments may have a unique marker associated with it (e.g., a marker such as a QR code printed on the clothing or a signal being broadcast that, when received by a user for example, identifies the clothing and provides data, etc.) as well as having a contour marking around the outline of the clothing that, as discussed herein, is capable of defining an area of the clothing that can be replaced with augmented reality fashion data. Hereinafter, reference to “article of clothing” or a “shirt” or any other type of clothing should be understood to mean any article of clothing worn by a person and utilized according to the techniques described herein.

Various approaches discussed herein enable augmented reality fashion (e.g., enabling an article of clothing worn by a person in the real world to appear different when viewed in augmented/virtual reality) by obtaining original image data (e.g., through a camera of a computing device, through a virtual reality wearable device, etc.) that includes a representation of an article of clothing actually being worn by a person represented in the image data. In various embodiments, this original image data may be multiple frames of video data. The article of clothing will take on a particular appearance in the augmented reality, which is enabled in various embodiments by the actual article of clothing having certain features such as a unique marker and a contour marking, the operation of which will be described more fully herein.

According to various embodiments, a user may view a person wearing the clothing and see an “augmented” version of the clothing specified by the wearer, which may be a new color or pattern, or even what will appear to the user to be a new piece of clothing entirely. Augmented reality according to approaches described herein may be considered as identifying a masking area in image data, for example constrained by a reference marker and/or contour marking, that is inlaid with substitute image data that is provided to the display device in which the image data is being displayed. For example, a person can specify that a user who views them in augmented reality, according to the various embodiments described herein, will see a blue shirt in the image data being presented to them, even though the actual shirt being worn by the user may be white or some other color. In various embodiments, augmented clothing according to the techniques described herein may appear to show a picture, text, video, play audio, or even 3-D objects that offer various perspectives of the object in augmented reality according to a corresponding perspective of a viewer in relation to the marker on the shirt, for example. In some embodiments, a third party may offer certain benefits to a user who allows those third parties to display image data on their virtual “clothes,” for example a credit or other type of payment, free virtual clothes, etc. Placement of his third party data may correspond to a specific reference marker, contour marking, or some other data.

For example, the techniques described herein would allow for a user to “virtually” change their clothes several times a day with the click of a button. A user may select various “virtual” clothes (e.g., through an app or a website, etc.), such as by logging into their account and selecting from a plethora of virtual clothing options. In various embodiments, other people (e.g., fashion experts, celebrities, etc.) may offer virtual clothing “collections” that can automatically be selected for a user, for example at various days, times, locations, etc. In various embodiments, a user can control what any random person might see them wearing in augmented reality, as well as specific users whose identities may be determined, for example by determining an identifier associated with a device that is linked to a person's identity. A user may assign specific virtual “outfits” to be seen by certain people or groups of people.

A determination may be made regarding a location of the reference marker that is represented in the image data, as well as a boundary of the clothing in the image data that substantially corresponds to the contour marking on the clothing. Using some combination of these and/or other features, a replacement area in the original image data is determined that is based on various data, such as the location of the reference marker as it relates to the boundary of the clothing in the image data. For example, the location of the reference marker is found in the image data and used as part of the approach to finding the edges of the boundary, such as by starting at the reference marker and working outward towards to periphery of the clothes as represented in the image data. Once a replacement area is determined, substitute image data, such as might be associated with the reference marker, is identified and used to create modified image data that corresponds to an overlay of the substitute image data on the original image data, the substitute image data being positioned in the replacement area and displayed substantially in real time as a user is viewing the person wearing the clothing.

Various conventional approaches can attempt to superimpose an article of clothing onto a representation of a person's body, but such approaches can be computationally expensive and not highly accurate. Various other applications, processes, and uses are presented below with respect to the various embodiments, each of which improves the operation and performance of the computing device(s) on which they are implemented, as well as improving the technology of virtual and/or augmented reality.

FIGS. 1A and 1Billustrate example situations100,130where a user102can view augmented reality fashion, in accordance with various embodiments. In the example ofFIG. 1A, a person110is wearing an article of clothing112(in this example, a shirt, although in various embodiments any type, cut, fit, or style of clothing may be used). The shirt112in the example ofFIG. 1Ahas several markings. A contour marking114appears on the shirt112, for example to define a shape or periphery of the shirt112. According to various embodiments, contour marking114may be used as part of an approach to determine a boundary of an article of clothing represented in image data, such that a replacement area in the image data may be determined. If other image data (e.g., substitute image data or modified image data, as described herein) is designated or otherwise selected to appear on the image of the article of clothing, then the other image data may fill the entire replacement area so that it appears as if the boundary defined by the contour marking on the article of clothing has been filled (e.g., replaced, overlaid, etc.) with the other image data, including covering the contour markings. As described herein, there may be multiple contour markings114defining an outline of the shirt112as viewed from various angles, and contour marking114may take up more or less of the shirt112as illustrated in the examples ofFIGS. 1A-1B. Additionally, shirt112may comprise a “call to action” element, such as text, a logo, code, etc., that instructs a viewer in the “real world” to view the wearer through a device that provides the augmented reality fashion experience.

In an embodiment, a contour marking defines a shape, for example by determining various points along the contour marking and matching those points to actual or virtual points defining an article of clothing, for example. The contour marking may be considered as a continuous shape descriptor or as a discrete number of points along a periphery of an article of clothing, which when matched or otherwise compared to points along a periphery of prospective article of clothing, may allow the outer boundary or inner area to be reliably determined.

In the example ofFIG. 1A, contour marking114is visible on the article of clothing112, in the example being a dark or black coloring defining all or part of the outline of the article of clothing112. For example, a dress implementation of the article of clothing may have a contour marking representing an outline of the dress around areas of the dress that represent the boundary around the periphery of the dress. The contour marking114may be dependent on the view of the article of clothing112. For example, a contour marking for a shirt viewed from the side will be in a different location on the shirt than a contour marking meant to be viewed head-on, as in from the front. In some embodiments, there may be multiple contour lines for different views. For example, one contour line may represent the outline of an article of clothing when viewed from the front (or back), while another contour line on the article of clothing may be intended to represent the outline when viewed from the side. In some embodiments with multiple contour lines, each contour line representing an outline from a different view perspective may be differentiated in some way; for example, with different colors or patterns, such that automated image analysis or interactive selection may discern which contour line to use in a given situation. Contour lines may also be printed on the article of clothing in different type of ink or other substance. For example, a contour line may be created using dye, ink, or other substance that appears invisible to the naked eye, but appears when viewed in an image, for example when using special image processing, or a special camera setting, for example to detect UV ink or dye. This technique may also be used to differentiate between multiple contour lines.

In some embodiments, data representing locations on an article of clothing where a contour marking would be, or had been, may be stored and represented in some way. For example, if data representing a contour marking were associated with a reference marker or some other marking, as discussed further herein, then the data could be accessed and a boundary determined, such that a replacement area could be determined and other image data accurately placed onto the shirt represented in the image data.

The shirt112in the example ofFIG. 1Aalso has a reference marker116displayed on the front of the shirt112, although the location and number of the reference marker(s)116may vary according to the embodiment. While the reference marker116ofFIG. 1Ais represented as a QR code, other examples of reference markers116may be used; for example, fiducials, text, wireless signals such as those emitted by a beacon or other type of transmitter, either worn on, in, or woven into the shirt112, etc. Any type of marking may be used, for example one that may be recognized and/or translated into data by a computer, and the reference marker(s)116may comprise a unique identifier, either individually or as a collection. In an embodiment, a reference marker may comprise an article of clothing; for example, a wearer may have a shirt with a car on the front. By taking a picture of the shirt and providing that image to a system that performs the recognition of the reference markers, as described further herein, once that shirt is recognized, for example by a viewer acquiring original image data through a camera on his cellphone, then the shirt itself operates as a reference marker (e.g., an identifier). This may also be the case with a sub-portion of the shirt; for example, the entire shirt with the car may have to be “recognized” in order to operate as a reference marker, or simply a portion of the shirt, such as the car. A user may be provided an interface on a device, such as the device used to take the picture, where the user can select the portion of the shirt to use as the reference marker. For example, the user may take a picture of the shirt with the car, and select the car, for example by using an interface element to draw a selection marker around the car. The reference marker116may be used in various embodiments to access data about the article of clothing112; for example, dimension data (e.g., size, inseam, neck circumference, etc.) about the article of clothing112and data describing the type of article of clothing112(e.g., shirt, dress, leggings, etc.). According to various embodiments, reference marker116may provide data about a particular user, for example to which the shirt112is linked, for example to reference a particular user account of a user110who is wearing a shirt112with a reference marker116associated with the user's110account.

In the example ofFIG. 1A, a viewer102is capturing image data of person110who is wearing the article of clothing112containing the reference marker116and contour marking114. Viewer102is using a portable computing device104with a camera device106to capture image data contained in the capture angle108of the camera106. For example, viewer102may be taking a picture or video data of person110, or simply be viewing image data of person110on the screen of the device104as it is captured in real-time by the camera106.

In the example ofFIG. 1B, image data134being displayed on the screen of the computing device104of viewer102is illustrated. As will be discussed further herein, the shirt112being worn by person110and represented in the image data134has been augmented, such that the image data134displayed on the screen shows a different shirt136being worn by person110, including substitute image data138that is overlaid on the image data134and positioned such that the original shirt112appeared to be replaced, including the reference marker116and contour marking114. In this example, the reference marker116is translated to data that identifies person110, for example person's account at a website where person110has selected substitute image data138to be displayed (e.g., overlaid, etc.) onto image data being displayed of person110. The substitute image data138may comprise text, images, patterns, colors, etc., so while the person110walking around in the real world is wearing a shirt112with a reference marker116and contour markings114, once image data is captured of person110and the reference marker116is identified, the substitute data138associated with the reference marker116may be retrieved and superimposed on a display of the image data134, using the reference marker and/or the contour marking114to position the modified image data138in the image data134.

Additionally, in various embodiments, the identity of the viewer102may be determined, such that a particular portion of substitute image data138may be linked to the viewer102, such that the identity of the viewer102is transmitted to a server along with the reference marker116, and the particular portion of substitute image data138linked to the viewer102is displayed. In the example ofFIG. 1B, person110may know viewer102to also be a fan of a certain team that just won a game. Therefore, person110specified that, if viewer102(or a device linked to viewer102) attempts to acquire augmented image data associated with person's110reference marker116, then the substitute image data138will be a message that “we won,” referring to the team of which they are both fans. Various embodiments in which other criteria are utilized to determine substitute image data

While the examples ofFIGS. 1A-1Bare illustrated in black and white, any color, shade, tint, or pattern may be used, such that can be discerned either visually or computationally; additionally, various types of markings may be utilized for the contour marking114, such as those having varying thicknesses or alternating placements, as well as being a continuous marking around a periphery or outline of the article of clothing or only partially present. For example, the contour marking114could be a dashed marking, or be present at certain places around the article of clothing, for example at particular points (e.g., dots at the bottom corners, sleeve bottoms, mid-neckline, etc.) which could be sufficient to ascertain the outline or shape of the article of clothing.

FIG. 2illustrates an example approach200for finding a replacement area on an article of clothing, in accordance with various embodiments. In the example200ofFIG. 2, an article of clothing202(a shirt) is illustrated along with examples of a reference marker206and contour marking204that in the example ofFIG. 2are printed on the shirt202, although as discussed herein, various embodiments in which reference marker206and contour marking204comprise different forms are envisioned. According to various embodiments, reference marker206is used along with contour marking204in order to determine how to position substitute image data in order to make substitute image data appear as if it is being “worn” in augmented reality by the person wearing the shirt202. In one example, a process for locating contour marking204of shirt202would begin by locating reference marker206and proceed outward208until contour marking204has been located. In another example, if a person chooses a plaid pattern as substitute image data, then an embodiment would locate reference marker206and use it as a starting point to overlay the pattern outward208until a boundary in the image data corresponding at least in part to the contour marking204is reached, or in some embodiments, until the contour marking204is reached in the image data.

According to an embodiment, a reference marker206may not necessarily comprise a visible marking on a shirt202, as illustrated inFIG. 2. One or more sensors210could be utilized, for example to broadcast a signal containing data that identifies a particular person, account, etc. One or more sensors210may be worn on the shirt202, be woven into the shirt202, or otherwise be associated with the shirt202. In some embodiments, sensors210may be used to determine stress on the shirt202, such that areas of the shirt202that are stretched or wrinkled due to not being stretched may be identified and utilized to assist in placement/positioning of substitute image data.

In addition to potentially acting as a data source as described herein, reference marker206may be used as an “anchor” or reference point, which when combined with the contour marking204, provides a more accurate approach to analyzing a replacement area of a shirt202. For example, a process of analyzing the shirt202to determine where to position substitute image data may begin by locating/recognizing a placement of reference marker206, and then proceeding to ingesting/recognizing/analyzing the image data representing the shirt202until reaching the contour marking204, thereby giving an indication that the shirt has been completely and accurately ingested into the analyzing process.

According to an embodiment, reference marker206may provide sizing or other dimension data associated with the shirt202, as well as data regarding any characteristic of the shirt202such as cut, fit, fabric, etc., which may be utilized to in the process for analyzing and determining where to position substitute image data, as described earlier. A location of the reference marker206may also be relative to contour markings204for different sizes of shirts202.

In an embodiment, reference marker206may also provide data indicating a type of shirt, as well as cut, fit, etc. of a shirt that should be used in the augmented reality. For example, a reference marker may indicate that the shirt being worn in real-life is a long-sleeve shirt; therefore, the substitute image data should be positioned with that in mind, as well as patterns, colors, etc. taking the long sleeves into account. If the reference code indicates a short-sleeve shirt where the sleeves end at the mid-bicep, but the substitute image data used is that of a long-sleeve shirt, then the view of the wearer in the augmented reality will be “off” regarding the person's arms below mid-bicep.

FIG. 3illustrates an example approach300for substituting image data on an article of clothing in an augmented reality, in accordance with various embodiments. It should be understood that reference numbers may be carried over between figures for similar components for purposes of explanation, but that such use should not be interpreted as a limitation on the various embodiments. In the example300ofFIG. 3, person110is wearing shirt112that has a reference marker116and contour marking114. Image data of person110is captured, and the reference marker116is analyzed, such that it serves as a unique identifier for person110. A query may be made, either on a device or remotely, for example to a server or cloud computing environment, such as discussed later with reference toFIG. 10, and substitute image data320is accessed. For example, substitute image data320may comprise a data repository of numerous potential portions/instances of substitute image data320, all or some of which may be associated with person110through reference marker116(e.g., an account associated with person110through reference marker116, etc.), although in various embodiments, persons other than a wearer of a shirt about to be “augmented” may be associated with substitute image data320. In this example, a particular piece of substitute image data310is identified in the substitute image data repository320, such as using techniques discussed further herein. Substitute image data310is then transmitted to the device on which the image data corresponding to the person110wearing the shirt112is being viewed (e.g., from a data store on the device or from a remote location over a network to a device, etc.), and is used to generate an augmented reality version of the shirt330(e.g., modified image data, etc.) that displays the substitute image data332as an overlay (or replacement, merge, etc.) for the original image data captured by the device.

FIGS. 4A-4Billustrate an example situation400,450for presenting different augmented reality articles of clothing to different users, in accordance with various embodiments. In the example400ofFIG. 4A, person110is wearing shirt112with a reference marker116and contour marking114. Two different viewers402,410are capturing image data (e.g., still images and/or video data) of person110with cameras406,412of their devices404,414, which in the example ofFIG. 4Aare mobile computing devices such as smartphones and/or tablets, but in various embodiments could comprise other types of computing devices, as discussed further herein. In the example ofFIGS. 4A-4B, identities associated with the viewers402,410are determined, such as usernames, social graph identifiers, account information, etc. By determining an identity associated with a viewer of image data that is to be augmented according to the techniques described herein, different substitute image data may be selected to be displayed to different viewers, for example based at least in part on the viewer's identity.

In the example450ofFIG. 4B, the identities of the viewers has been determined and different substitute data is being delivered to the devices406,412to augment the original image data. On the device406of the first viewer402(not pictured), the reference marker116is identified in the original image data, and a query is transmitted474over a network470, for example to a substitute image data store476that contains two portions of substitute image data478,480. Based on an identity associated with the first device406, a first portion of modified image data480is selected and transmitted back474to the device406, so that it can be displayed452such that the particular portion of substitute data476augments the original image data so it appears on the display of the device as if the person110is wearing the shirt that comprises the particular portion of substitute data476.

Likewise, second viewer410(not pictured) is capturing image data of person110on his device412, and based on the reference marker116and an identity associated with the device412(or with the viewer, etc.), a query is sent to substitute image data store476and the other portion of substitute image data478is selected and sent back to the device412for display460. In this example, two devices406,412, each capturing image data of the same person110, the same shirt112, and the same reference marker116end up displaying different augmented reality fashion452,460.

According to various embodiments, any number or type of constraints and/or criteria may be utilized in a determination of which substitute image data is to be selected and sent to a device for display (although in some embodiments, all of the capturing, processing, and selection of data is performed on a device). For example, if a person points their cellphone camera (or views them through a VR wearable device, etc.) at a particular person wearing a shirt in accordance with the approaches described herein, then the person wearing the shirt can have control over what that person sees in the augmented reality fashion displayed on their device, for example if an identity of the viewer can be established. For example, a person can specify that all people viewing him today that send a query associated with the person's reference code (e.g., that is displayed on the shirt being worn by the person) before 4 PM will see a blue striped dress shirt, and after 4 PM, the viewers will see the person wearing a Hawaiian shirt. The person could also choose different people or groups of people to receive different substitute image data, as described earlier. A constraint could also include relationship data, for example on a social network or social graph. For example, a person may designate all people that he has labeled “friends” will see a t-shirt with a logo of his favorite band, while people he has labeled “business” will see a more conservative shirt. The control may be as granular as desired. For example, a person could have a private joke with one of their friends, and designate it such that when that friend is viewing them in augmented reality, that the joke be displayed to them, but only to them.

FIG. 5illustrates an example situation500where a user can view augmented reality fashion, in accordance with various embodiments. In the example500ofFIG. 5, a viewer502is looking at a web page504, for example a person's profile510on a social network web site. The profile owner may have posted a picture of himself512wearing an article of clothing112as described herein, with a reference marker116and contour marking114. The device on which the web browser504is executing can recognize the reference marker116in the image data512, and using the techniques described herein, cause a substitute portion of data508to be selected and displayed506along with the original image data512, such that the augmented reality image506makes it appear to the viewer502as if the person is wearing the substitute image data instead of the shirt112. According to another embodiment, viewer502will see the augmented reality image506when he looks at the web page504through a device that captures image data, as described herein. In an embodiment, an additional program, such as a browser plug-in, may be utilized to provide the present functionality on any website, while some websites may be able to provide native support for the present techniques; for example, depending on where the original image data, substitute image data, and/or device are stored, executing, and/or connected.

FIG. 6illustrates front600and side650views of an example situation where a user can view augmented reality fashion, in accordance with various embodiments. InFIG. 6, a person is wearing a shirt630including a reference marker632. In the front view600, image data is being captured by a device602such that the device is straight-on to the front of the shirt630, where the reference marker632is positioned. As a result, the image data of the shirt610displayed on a screen604of the device is a straight-ahead view. In the example ofFIG. 6, the substitute image data comprises a 3-D object612,622that has multiple view perspectives. In the front view600, one perspective view of the 3-D object is displayed612as if it were floating in front of the augmented shirt610. In the side view650, the camera of the device602is capturing the shirt630from a side view, such that the perspective of the reference marker in the image data is representative of the camera view perspective when compared to the front view600. By identifying the view perspective based at least on the recognized angle of the reference marker632, then the appropriate view of the 3-D object may be selected as the substitute image data. In the side view example650, this results in a different view of the 3-D object622being displayed floating in front of a side view of the augmented shirt620. In alternate embodiments, different reference markers, as well as one or more contour markings, could be utilized to indicate different angles and/or perspectives of view, such that an identification of a particular reference marker could indicate which perspective of the 3-D substitute data to select and display.

According to various embodiments, a person could capture image data in a 360 degree field around a person wearing an article of clothing in accordance with the techniques described herein, and the view of a 3-D substitute image data would be selected based on the corresponding view of the device capturing the image data, for example based on a determination of a position/angle/skew/tilt/etc. of one or more reference markers, contour markings, wireless sensors, etc., as well as a distance from the marker, which could be used for example to determine how large to make the 3-D substitute image data and how far in front of (for example) the shirt the 3-D substitute data should “float.” For example, if a person designates a 3-D image of a sports car to be displayed in an augmented reality view of his shirt, then a viewer viewing the shirt head-on through their device would see, depending on various criteria designated by the person, a head-on view of the sports car. As the viewer changes their perspective, for example by circling around one side on the person, who remains stationary, then the changed perspective would be determined and the substitute image data changed accordingly, such that the 3-D substitute image data would have a perspective corresponding to the perspective of the viewer. In some embodiments, the 3-D substitute image data may be made to appear to move around the person wearing the shirt; for example, using various reference markers, contour markings, and/or other perspective data, etc. For example, the sports car of the above example may be made to “drive” around the person in augmented reality. Other types of animation and/or sounds, etc. may provided.

FIG. 7illustrates an example augmented reality device700that can be utilized in accordance with various embodiments. Various other types of devices, such as smart glasses, goggles, and other augmented and/or virtual reality displays and devices can be used as well within the scope of the various embodiments. According to various embodiments, augmented reality and virtual reality may provide different experiences. For example, virtual reality may replicate an environment that simulates a presence in a virtual world, while augmented reality viewing devices are wearable (or portable) devices that add information to what the viewer sees in the real world, for example with transparent or heads-up displays or other displays and/or overlays that allow for the projection of digital images and other data into a person's view of the real world. The augmented reality device700may take a form of a traditional virtual reality headset or an augmented reality device, and the example device700is not limited to one or the other. In this example, the device includes a housing702made of a material such as plastic with a polymer lip714or other such portion intended to contact the user's face in order to provide for comfort of the user as well as providing a relatively light-tight seal to prevent extraneous light from passing to the user's eyes while wearing the device. The example device also includes a strap716or other such mechanism for securing the device to the user's head, particularly while the user's head704is in motion. The example device includes a left eye display screen708and a right eye display screen, although as mentioned in some embodiments these can be portions of a single display screen or arrays of multiple screens, or holographic displays, among other such options. In some embodiments a single display element will be used with respective convex lenses for each eye and one or more separation elements that limit the field of view of each eye to a designated portion of the display. The device will typically include display circuitry718, as may include memory, one or more processors and/or graphics processors, display drivers, and other such components known or used for generating a display of content. There can be a single set of circuitry for both displays706,708or at least some of the components can be duplicated for each display such that those components only provide for display of content on one screen or the other. The display screens can be any appropriate type of display, such as an AMOLED or LED display with sufficient refresh rate for augmented and/or virtual reality applications. The device includes one or more motion and/or orientation sensors710, as may include at least one accelerometer, magnetometer, gyroscope, electronic compass, inertial sensor, and/or other such sensor for providing data about rotation, translation, and/or other movement of the device. The motion and/or orientation data can be used to determine the appropriate point of view (POV) from which to render the current scene of content. The example device also includes at least one communication component712, such as a wired or wireless component for transmitting data over a protocol such as Bluetooth, Wi-Fi, 4G, and the like. The communication component can enable the device700to communicate with a computing device for purposes such as obtaining content for rendering, obtaining additional input, and the like. The example device can include other components as well, such as battery or power components, speakers or headsets, microphones, etc.

The example device700can also include one or more cameras720,722or other image capture devices for capturing image data, including data for light reflected in the ambient or infrared spectrums, for example, which in various embodiments may be used to “see” the real world on which augmented reality may be superimposed, as well as discern features on an article of clothing that may otherwise be invisible to the naked human eye (e.g., ink used to print a contour marking on an article of clothing, the ink only being visible in the ultraviolet spectrum, etc.). One or more cameras can be included on an exterior of the device to help with motion tracking and determining environmental conditions. For example, locations of light sources, intensity of surrounding ambient light, objects or persons nearby, or any of various other objects or conditions can be determined that can be incorporated into the augmented and/or virtual reality scene, such as to make the lighting environmentally appropriate or to include things located around the user, among other such options. As mentioned, tracking the motion of objects represented in the captured image data can help with motion tracking as well, as rotation and translation data of surrounding objects can give an indication of the movement of the device itself.

Further, the inclusion of one or more cameras720,722on the inside of the device can help to determine information such as the expression or gaze direction of the user. In this example, the device can include at least one IR emitter724, such as an IR LED, that is capable of emitting IR radiation inside the device that can be reflected by the user. IR can be selected because it is not visible to the user, and thus will not be a distraction, and also does not pose any health risks to the user. The IR emitter724can emit radiation that can be reflected by the user's face and detected by one or more IR detectors or other image capture elements720,722. In some embodiments the captured image data can be analyzed to determine the expression of the user, as may be determinable by variations in the relative locations of facial features of the user represented in the captured image data. In some embodiments, the location of the user's pupils can be determined (as discussed elsewhere herein), which can enable a determination of the gaze direction of the user. The gaze direction of the user can, in some embodiments, affect how objects near to, or away from, the center of the user's field of view are rendered.

FIG. 8illustrates an illustrates an example process800for augmented reality fashion, in accordance with various embodiments. It should be understood that, for this and other processes discussed herein, there can be additional, fewer, or alternative steps, performed in similar or alternative steps, or in parallel, within the scope of the various embodiments unless otherwise stated. InFIG. 8, image data of an article of clothing is received802, for example by a camera of a computing device that is capturing the image data of a person wearing the article of clothing in real-time. An identifier, such as a reference marker, is identified804in the image data. A contour marking of the clothing is identified and used to determine a boundary806of the article of clothing. In various embodiments, the contour marking serves as the boundary. A replacement area in the image data is determined808, the replacement area being determined by analyzing a relation of the identifier to the boundary of the clothing. For example, by starting at the identifier and working outward to the boundary identified in the image data, the replacement area may be defined.

A viewer identity is determined810, for example by analyzing data associated with the device being used to capture the image data and that is transmitted along with data associated with the identifier. A set of potential substitute image data is obtained that corresponds to the identifier812, and one or more portions of substitute image data are selected and obtained814from the set of potential substitute image data, for example based on the viewer identity. Modified image data is generated812, which according to an embodiment comprises a placement of the substitute image data on the original image data (i.e., the image data captured of the person wearing the clothing), and the placement of the substitute image data corresponds to the replacement area. The modified image data is displayed814, for example in real-time on the viewer's device. In various embodiments, the substitute image data812may be displayed in locations or positions other than to appear “on” the clothing; for example, one piece of substitute image data may comprises a pumpkin that appears on a person's head, while another piece of substitute image data may comprises a Halloween-themed shirt or text or video, etc., and the modified image data will reflect each. Any number of pieces/portions of substitute image data may be used in the creation of modified image data812.

FIG. 9illustrates an example process900for augmented reality fashion, in accordance with various embodiments. In the example ofFIG. 9, image data of a person wearing an article of clothing, such as a shirt, is received902, for example by a camera of a viewer's computing device. The image data may comprise multiple frames of image data, for example a recorded video or real-time image data streaming from the camera of the device. Substitute image data is determined904, for example based on a reference marker associated with the clothing. Modified image data for a frame of the image data is determined906, and is displayed along with the current frame908of image data being displayed on the device, according to the techniques described herein.

A detection of movement is made910, for example by comparing a position of the reference marker and/or the contour marking in one frame of image data to a subsequent frame. In response to detecting movement of the clothing in the image data, new substitute image data is determined for the new frame of image data912, if necessary. Based on the new substitute image data and the new position of the clothing in the image data, modified image data is generated for the new frame914of image data, and is caused to be displayed916for the new frame of image data.

FIG. 10illustrates an example environment1000that can be used to implement aspects in accordance with various embodiments. InFIG. 10, a client computing device1002can submit content and/or a request for content across at least one network1004to be received by a resource/content provider environment1008. According to various embodiments, a client computing device1002may comprise a laptop or desktop computer, a set-top device, etc., as well as various types of mobile devices, such as cellphones, tablets, virtual reality headsets, and wearable devices (e.g., glasses, contact lenses, etc.). The network(s) can include any appropriate network, such as the Internet, a local area network (LAN), a cellular network, an Ethernet, or other such wired and/or wireless network. The resource provider environment1008can include any appropriate resources for providing content from a resource provider, as may include various servers, data stores, and other such components known or used for providing content from across a network (or from the “cloud”).

In this example, a call received to the resource provider environment1008can be received by an interface layer1010of the environment. As known for network environments, the interface layer can include components such as interfaces (e.g., APIs), load balancers, request and/or data routers, and the like. If the request is a request for content, such as for content for a page to be displayed in an application (e.g., browser), information for the request can be directed to one or more content servers1012, which can obtain the content from a content data store1014or other such repository to be sent back across the network(s) to the computing device. In some embodiments, information for the request might also be compared against user data in a user data store1016or other such location do determine, for example, an identity of the user, whether the user has access rights to that content, etc. In one example, the content can include a plurality of images to be displayed as part of a set of search results, although various other types of content and uses for images can be utilized as well within the scope of the various embodiments.

In some cases, a request received to the content provider environment1008might be from another entity, such as a third party image provider1006. As discussed previously, such providers may provide images to be displayed to users along with the served content. The interface layer can determine the type of request and cause information to be forwarded to an image processor1018or other such component, which in some embodiments can cause the images to be stored to an image data store1020at least temporarily. Since it may be desirable to manipulate at least some of the images before being presented for display, as discussed herein, the image processor can process at least some of the images before causing those images to be stored in the content repository1014, for example, to be presented for display to a user or otherwise utilized.

The content provider environment1008can also include an image analysis component1022, system, or service that is able to analyze images using approaches discussed herein. These can include, for example, images received from third party suppliers, images stored in the content database1014, or images received from a user, among other such options. These images thus can also be stored in the content database1014, or another appropriate data store, at least for augmentation purposes. The resource provider environment1008also can include at least one image augmentation component1024, system, or service, which can receive data about images from the image analysis component1022and attempt to receive data stored in the content database1014, whereby the results of the augmentation can be passed to the content server1012for transmission to the requesting computing device1002.

According to an embodiment, the techniques described herein may be performed solely or partially on a client computing device1002, without utilizing the resource provider environment1000for some or all portions of the techniques described herein. For example, a client computing device1002may perform various aspects of the techniques described herein without at least some portions of data leaving the client computing device1002, thereby ensuring privacy of data such as the image data or identity data described herein.

Once the data associated with the article of clothing, including data regarding a contour marking and/or reference marker, among other potential identifiers or markings contemplated by one of ordinary skill in the art, then a user may receive substitute image data used to provide an augmented reality display, for example on the same cellphone, or on a device under their control (e.g., a wearable device such as a VR headset), such that data used to provide the augmented reality may be shared between the devices without the need to communicate the data to a third party, for example over the Internet. All augmentations of the prospective article(s) of clothing may then be determined on the user's device, with resources only transferred from a third party (e.g., a resource provider environment) on an as-needed basis, such as product images, account information, HTML pages, etc., thereby securing the device and any private body data from unwanted transmission.

FIG. 11illustrates an example computing device1100that can be used in accordance with various embodiments. Although a portable computing device (e.g., a smart phone, an electronic book reader, or tablet computer) is shown, it should be understood that any device capable of receiving and processing input can be used in accordance with various embodiments discussed herein. The devices can include, for example, desktop computers, notebook computers, electronic book readers, personal data assistants, cellular phones, video gaming consoles or controllers, wearable computers (e.g., smart watches or glasses), augmented and/or virtual reality wearable devices, television set top boxes, and portable media players, among others.

In this example, the computing device1100has a display screen1104and an outer casing1102. The display screen under normal operation will display information to a user (or viewer) facing the display screen (e.g., on the same side of the computing device as the display screen). As discussed herein, the device can include one or more communication components1106, such as may include a cellular communications subsystem, Wi-Fi communications subsystem, BLUETOOTH® communication subsystem, and the like.FIG. 12illustrates a set of basic components of a computing device1200such as the device900described with respect toFIG. 9. In this example, the device includes at least one processor1202for executing instructions that can be stored in a memory device or element1204. As would be apparent to one of ordinary skill in the art, the device can include many types of memory, data storage or computer-readable media, such as a first data storage for program instructions for execution by the at least one processor1202, the same or separate storage can be used for images or data, a removable memory can be available for sharing information with other devices, and any number of communication approaches can be available for sharing with other devices. The device typically will include at least one type of display element1206, such as a touch screen, electronic ink (e-ink), organic light emitting diode (OLED) or liquid crystal display (LCD), although devices such as portable media players might convey information via other means, such as through audio speakers. The device can include at least one communication component1208, as may enabled wired and/or wireless communication of voice and/or data signals, for example, over a network such as the Internet, a cellular network, a Wi-Fi network, BLUETOOTH®, and the like. The device can include at least one additional input device1210able to receive conventional input from a user. This conventional input can include, for example, a push button, touch pad, touch screen, wheel, joystick, keyboard, mouse, trackball, camera, microphone, keypad or any other such device or element whereby a user can input a command to the device. These I/O devices could even be connected by a wireless infrared or Bluetooth or other link as well in some embodiments. In some embodiments, however, such a device might not include any buttons at all and might be controlled only through a combination of visual and audio commands such that a user can control the device without having to be in contact with the device.