Augmented reality data exchange

A technology is described for evaluating object data associated with an object represented in an augmented environment. An example method may include detecting an object located within range of a proximity of a sensor included in a device configured to display object data for the object in an augmented environment display. The object data associated with the object may be requested and the object data may be authenticated using an authenticating authority that certifies that the object data is valid and/or by determining that an object attribute represented by the object data corresponds to the object attribute represented by sensor data obtained from the sensor. After the object data has been authenticated, the object data, or a portion of the object data may be displayed in the augmented environment display generated by the device.

BACKGROUND

The advent of augmented reality technologies has resulted in the ability to augment a view of a physical environment with computer-generated sensory input. For example, augmented reality can be used to add graphics, sounds, haptic feedback, and geographical location data to the natural world as experienced by a user of a computing device. The augmentation of a physical environment may blur the line between what a user may experience as real and what may be computer-generated by enhancing what a user may see, hear, feel, and even smell.

Augmentation of a physical environment may be performed at view time, namely, at the time a user is viewing the physical environment using a computing device. Augmented reality may include combining semantic context with environmental elements, such as sports scores displayed on augmented reality glasses during a sports match or product information displayed when a product is viewed.

DETAILED DESCRIPTION

A technology is described for a data exchange protocol used to validate and filter data associated with extrinsic objects displayed in an augmented environment that may include both physical objects and computer generated objects. An AR (Augmented Reality) device may be configured to include one or more sensors (e.g., a camera, microphone, GPS (Global Positioning System) module, accelerometer, wireless communication module, as well as other sensors) that may be used in generating and displaying an augmented environment on the AR device. An AR device may include: hand held devices, mobile devices, head-mounted devices (e.g., eyeglasses, contact lenses, and virtual retinal displays), HUDs (Head-Up Displays), and other devices configured to display an augmented environment.

Physical objects may be configured to be detectable by an AR device sensor when the AR device is in proximity of the physical object. For example, a physical object may be an electronic device that includes an interactive device, such as a RFID (Radio Frequency Identification) module, a BLUETOOTH module, a WI-FI module, NFC (Near-Field Communication) module, and/or like interactive devices. Also, a physical object may be a non-electronic object associated with an interactive device, such as an RFID tag, beacon (e.g., WI-FI or BLUETOOTH), optical code (e.g., bar code or QR (Quick Response) code), or like devices. A physical object (via an interactive device) may provide an AR device with object data that includes object related information that can be included in an augmented environment display generated by the AR device. As an illustration, an office building may have an interactive device located at the entrance that provides an AR device with a building directory that can be displayed in an augmented environment generated by the AR device. Also, interactive devices may be associated with virtual objects (e.g., virtual advertisements, virtual assistants, etc.) that can be displayed in an augmented environment when an AR device is in proximity to the interactive device.

Object data that represents itself as being associated with an object within view of an AR device may be authenticated to determine the validity of the object data (e.g., whether the object data is actually associated with the object). By authenticating the object data, false or misleading information may be identified and discarded, preventing the information from being displayed in an augmented environment. Also, object data can be filtered based on user preferences. As such, a user can specify the types and amounts of information associated with objects that the user wants displayed in an augmented environment.

FIG. 1is a diagram illustrating an example of a method100for authenticating object data120obtained from an interactive device125claiming to be associated with an object110that is in proximity of a sensor included in an AR device105. As illustrated, an object110configured to include an interactive device125may be detected by a sensor included in an AR device105when the AR device105is in proximity of the object110that allows the sensor to detect the interactive device125. As a specific example, the object110may be an automobile that includes an interactive device125(e.g., WI-FI, BLUETOOTH, or NFC). The interactive device125included in the automobile may be detected by a wireless communication module included in the AR device105that is within a range of the interactive device125included in the automobile that allows the wireless communication module in the AR device105to detect the interactive device125included in the automobile.

After detecting the interactive device125associated with the object110, the AR device105may be configured to request object data120associated with the object110. The object data120may include object information130that can be displayed in an augmented environment115generated by the AR device105. The interactive device125may be configured to provide the object data120(or pre-data used to authenticate the identity of the object110) to the AR device105. In one example, the interactive device125may provide object data120stored on the interactive device125itself or object data120stored in an attached storage, and/or the interactive device125may provide a reference (e.g., a URI (Uniform Resource Identifier)) that may be used by the AR device105to obtain the object data120from a data service provider. The object data120provided by the interactive device125may include object information130that provides context for an object110included in an augmented environment115. The object information130may be displayed in the augmented environment115relative to the object110, thereby providing an AR device105user with context for the object110. As an illustration, object data120associated with an automobile may include automobile information that can be displayed relative to the automobile displayed in an augmented environment generated by an AR device105providing a user with information about the automobile.

Prior to displaying object information130included in the object data120in an augmented environment115generated by the AR device105, the object data120may be authenticated in order to provide a level of assurance that the interactive device125providing the object data120is actually associated with the object110. In one example, a request for object data120sent to an interactive device125may include a request for authentication data used to authenticate the object data120. In response to receiving the request, the interactive device125may be configured to send the authentication data to the AR device105, and the authentication data may be used by the AR device105to authenticate the object data120provided by the interactive device125.

One or more types of authentication may be used to authenticate object data120prior to displaying object information130included in the object data120in an augmented environment115. In one example, certificate authentication may be used to authenticate object data120. For example, an AR device105may request that an interaction device125provide a digital certificate135that may be used to authenticate object data120provided by the interactive device125. Illustratively, the digital certificate135may include information for a public key included in the digital certificate135, information about an interactive device's identity, and a digital signature of an entity (e.g., a certificate authority) that has verified that the digital certificate's contents are correct. The digital certificate135may be validated using a certificate authentication technique (e.g., validating the digital signature). In the case that the digital certificate135is successfully validated, the identity of the interactive device125and the object data120provided by the interactive device125may be considered valid.

In one example, a digital certificate135may include an embedded token that provides access to the object data120for a specified time period. For example, the token may provide access to object data120stored on the object110(e.g., computer memory or a storage device included in an object110), or the token may provide access to object data120managed by a data service provider. As an example, an AR device105may request a digital certificate135that includes an embedded token and after authenticating the digital certificate135, the AR device105may present the token to a data service provider within the specified time period and the data service provider may provide the object data120to the AR device105.

In another example, object data120may be authenticated by comparing object attributes included in the object data120with sensor data obtained from one or more of the sensors included in an AR device105to determine whether an object attribute of the object110represented by the object data120corresponds to the same object attribute represented by the sensor data. In other words, if object attributes represented by the object data120match object attributes represented by the sensor data, then the object data120provided by the interactive device125may be considered valid. In one example, object data120may include metadata describing various attributes of an object110. Sensors included in an AR device105may be used to collect sensor data, from which metadata may be obtained that describes the various attributes of the object110. One or more of the attributes included in the metadata obtained from the object data120may be compared to the attributes included in the metadata obtained from the sensor data, and if most, or at least a substantial number of the values of the corresponding attributes match one another, then the object data120may be considered valid.

As a specific example, object data120for an automobile may include metadata describing various observable features of the automobile, such as color, make, model, year, and state of the automobile (engine on/off, moving/stationary, geographic location). Sensors included in an AR device105may be used to collect sensor data associated with the automobile. For example, sensor data obtained by a camera may be analyzed to identify the color, make, model, and year of the automobile; sensor data obtained by a microphone may be analyzed to identify the state of the automobile (engine on/off, moving/stationary); and sensor data obtained by a GPS module may be used to identify a geographic location of the automobile reported by the automobile. Corresponding attributes represented by the object data120and the sensor data may then be compared to determine whether the values of the attributes match. For example, a color attribute represented by the object data120may be compared to a color attribute represented by the sensor data to determine whether the colors match. In the case that most, or a substantial number of the corresponding attributes match (e.g., color, model, make, and automobile state), the object data120may be considered valid.

In some examples, object attributes obtained from sensor data used to determine the validity of the object data120may be weighted according to a probability of misidentification of the attribute. For example, attributes having a lower probability of misidentification may be given more weight as compared to attributes having a higher probability of misidentification. As a specific example, a make and model of an automobile may be given greater weight due to a lower probability of misidentification (e.g., there may be a lower probability of classifying a truck as a sports car) as compared to a weight given to a color of the automobile due to a higher probability of misidentification (e.g., there may be a higher probability of misclassifying a blue car as a black car due to lighting conditions). Accordingly, weightings assigned to object attributes may be considered in authenticating object data120, such that object attributes assigned greater weightings may have a greater influence in determining the validity of the object data120. As an illustration, an automobile attribute set that includes matching make and model attributes and non-matching color attributes may be deemed to be valid based in part on the weightings assigned to the make, model, and color attributes, whereas an attribute set that includes matching color attributes and non-matching make and model attributes may be deemed to be invalid.

In another example, object data120may be authenticated using reputation data associated with the object data120that may be provided by a community of object data reviewers. A community of object data reviewers may include users (e.g., public and/or professional reviewers) that provide opinions regarding the object data120. For example, AR device users may submit reputation data (e.g., reviews) for object data120encountered by the users into a centralized data store (not shown). The reputation data may be made available to other AR device users and the reputation data may be used in evaluating the object data120when the users encounter the object110. The community of object data reviewers may include a user's social group that provides opinions regarding the validity of the object data120. The user may trust the opinions provided by the user's social group over the opinions of other users included in a community of object data reviewers. Also, social groups may rate whether object data may be relevant to the group (e.g., a group of automobile enthusiasts may rate object data120for automobiles). Illustratively, reputation data may provide a general consensus of object data validity (e.g., “valid”, “questionable”, or “not valid”), or the reputation data may provide a validity score (e.g., 1 to 10) for the object data120. A user may indicate a validity level or validity score that the user is willing to accept. In the case that reputation data associated with object data120satisfies a reputation data threshold set by a user, the object data120may be considered valid.

As already mentioned, multiple types of authentication may be used to authenticate object data120provided by an interactive device125. For example, object data120may be authenticated using various combinations of certificate authentication, attribute comparison authentication, and reputation data authentication, as described above. In using multiple types of authentication to validate object data120, a validation score may be assigned to the object data120, which can be used to determine whether or not to display object information130included in the object data120in an augmented environment115. In one example, a validation score may be calculated and assigned to object data120based in part on the results of individual authentications performed for the object data120. Illustratively, a validation score may be a “pass-fail” score, a numeric score, or some other type of score. Various methods may be used to calculate a validation score. For example, a validation score may be calculated by taking the sum, average, mean, or median of individual validation scores for individual authentications performed using separate authentication types. Also, individual validation scores may be weighted according to a type of authentication used to authenticate object data120. For example, certificate authentication may be given greater weight as compared to a weight given to reputation data authentication.

After assigning a validation score to object data120provided by an interactive device125, a determination may be made whether the validation score satisfies a validation score threshold that allows object information130included in the object data120to be displayed in an augmented environment115generated by an AR device105. A user may set the validation score threshold to coincide with the user's preferences. For example, if a user wants to see object information130that has a higher confidence of being associated with an object110, the user may set a higher validation score threshold. Whereas, if the user is less concerned whether the object information130is actually associated with the object110, the user may set a lower validation score threshold.

In one example, a user may specify the authentication types that are used to authenticate object data120. For example, a user may specify that certificate authentication be used in combination with attribute comparison authentication, and/or reputation data authentication. Also, a user may specify the parameters of an authentication method. For example, a user may specify that reviews submitted by a trusted social group be used for reputation authentication of object data120.

After the object data120has been authenticated, object information130included in the object data120may be displayed in an augmented environment115generated by an AR device105. The object information130may be positioned in the augmented environment115in relation to an object110that is displayed in the augmented environment115, such that the object information130may be associated with the object110.

In displaying objects110in an augmented environment115, in some examples, an object filtering parameter may be used to specify what types of objects110to display in the augmented environment115. For example, a user may control which objects110(physical and/or virtual) the user wants displayed. As a specific example, a user may block all or selected billboards (e.g., virtual billboards and/or physical billboards) from being displayed in an augmented environment115. Illustratively, physical objects may be filtered from an augmented environment display using object recognition software that may be configured to recognize a physical object in an image captured by a camera and remove the physical object from the image.

Also, a user may control what type or detail level of object information130to display in the augmented environment115using a data filtering parameter. Users may have the ability to choose from various levels of object information130to display. For example, a user may choose a summary level that includes basic details for an object110, or a detailed level that includes additional details associated with the object110. As an illustration, summary level object information130for an automobile may include a year, make, and model of an automobile, and detailed level object information130for the automobile may include additional details like engine specifications, performance specifications, and retail price.

A data filtering parameter may be applied globally to any object110that may be detected by an AR device105, and/or data filtering parameters may be applied to classes of objects110detected by the AR device105. For example, a user may set a global data filtering parameter to display summary or detailed object information130for all objects110detected by an AR device105, or the user may set individual data filtering parameters for classes of objects to display summary or detailed object information130.

FIG. 2is a block diagram illustrating an example system200used to implement a data exchange protocol and used to authenticate object data for inclusion in an augmented environment display generated by an AR device202. The system200may include an AR device202configured to detect objects204a-nthat are within a proximity of one or more AR device sensors that enables the AR device sensors to detect an object204a-nand request object data associated with the object204a-n.

In one example, the AR device202may be configured to detect an object204a-nand request object data associated with the object204a-nfrom the object204a-nitself and/or from a data service provider208. For example, an object204a-n(or an interactive device associated with an object204a-n) may be configured to store object data and transmit the object data to an AR device202in response to a request for the object data. Also, an object204a-nmay be configured to provide a reference that allows an AR device202to obtain object data from a data service provider208.

A data service provider208may include a network service (e.g., a content delivery network) that manages the storage and delivery of object data. The network service may be managed by an object owner, or the network service may manage the object data on the object owner's behalf. Various scenarios may call for obtaining object data from a data service provider208. As one example, supplemental object data may be obtained from a data service provider208and the supplemental object data may be included with object data received from an object204a-n. For example, detecting an interactive device associated with a bus stop may cause an AR device202to use an identifier included in bus stop information provided by the interactive device to obtain a current bus stop schedule from a data service provider208and display the schedule along with the bus stop information and the busses' progress in an augmented environment. As another example, an object204a-nmay be identified using sensor data (e.g., an image captured by a camera) and object data for the object may be obtained from a data service provider208. For example, a product bearing a manufacturer label or trademark may be identified in an image captured by an AR device camera and product information about the product may be obtained from a manufacture's data service and the product information may be included in the AR device's augmented environment display.

Object data obtained by an AR device202may be authenticated using the authentication methods described earlier. In one example, an AR device202may request authentication data (e.g., a digital certificate, object metadata, and/or reputation data) that can be used to authenticate the object data. The authentication data may be provided by an object204a-n, a data service provider208, an authenticating authority212, and/or a reputation data provider210.

An authenticating authority212may be an entity that certifies the validity of object data (i.e., certifies that object data claiming to be associated with an object204a-bis actually associated with the object204a-n). The authenticating authority212may be a trusted entity that vouches for the identity of an object data provider. In one example, the authentication authority212may be a certificate authority that issues digital certificates used to authenticate object data claiming to be associated with an object204a-n. A certificate authority may issue a digital certificate to an object data provider that contains a public key of the object data provider, a certificate authority signature, and the identity of the object data provider. A corresponding private key may not be made available publicly, but may be kept secret by the object data provider who generated or obtained the public/private key pair. The digital certificate may act as a confirmation or validation by the certificate authority that the public key contained in the digital certificate belongs to the object data provider identified in the digital certificate. In other words, the certificate authority via the digital certificate certifies that an object data provider is who they claim to be. If a user (or AR device202) trusts the certificate authority and can verify the certificate authority's signature included in a digital certificate, then the user can assume that a public key included in the digital certificate belongs to the object data provider identified in the digital certificate.

A digital certificate issued by a certificate authority may be stored on an object204a-nand the object204a-nmay present the digital certificate to an AR device202upon request. Alternatively, a digital certificate may be stored by a data service provider208. In response to a request from an AR device202, an object204a-nmay provide the AR device202with a reference (e.g., URI) that may be used to retrieve the digital certificate from the data service provider208.

A reputation data provider210may be an entity that collects and manages reputation data associated with object data. In one example, a community of object data reviewers may submit reviews for object data claiming to be associated with an object204a-n. The reviews may include user opinions about the credibility of object data based in part on observations made by the object data reviewers. For example, an object data reviewer may indicate that object data claiming to be associated with an object204a-ncorresponds to the object204a-n, or bears no or little relation to the object204a-n. The data object reviewers may use a rating system to review object data, such as a simple “pass-fail” rating system or a scaled rating system (e.g., 1 to 10 rating). Reputation data generated by the object data reviewers may be submitted to the reputation data provider210, and the reputation data provider may make the reputation data available to AR devices202.

As another example, an AR device202may be configured to submit reputation data to a reputation data provider210. In one example, reputation data submitted by an AR device202may coincide with a validation score assigned to object data that has been evaluated. For example, as described earlier, a validation score may be calculated and assigned to object data claiming to be associated with an object204a-nand the validation score may be used in a decision to include the object data in an augmented environment display. The validation score assigned to the object data may be sent to the reputation data provider210and the validation score may be made available to other AR devices202that encounter the object204a-nand are presented with the object data.

FIG. 3illustrates components of an example system300on which the present technology may be executed. The system300may include an AR device332configured to generate an augmented environment display that includes object data328associated with an object334detected by the AR device332. As illustrated, the AR device332may include a number of modules that may include an object detection module302, a data request module304, a data authentication module306, a data filtering module308, and a display module310.

The object detection module302may be configured to detect objects within proximity of the AR device332using one or more sensor devices320. Sensor devices320included in the AR device332may include, but are not limited to: cameras, microphones, GPS modules, accelerometers, networking devices (e.g., WI-FI, BLUETOOTH, NFC, and/or cellular module), and other types of sensors. An object334may include an interactive device that may be detectable by a sensor device320. For example, an interactive device included in an object334may include a networking device324(e.g., WI-FI, BLUETOOTH, NFC, and/or cellular module) and/or a passive wireless device326(e.g., an RFID tag or module). A sensor device320included in the AR device332may detect an interactive device included in an object334that is within a detection range of the sensor device320. After detecting an object334, the object detection module302may be configured to generate a notification indicating that the object334has been detected and send the notification to the data request module304.

The object detection module302may also be configured to detect when an object334is no longer in proximity of a sensor device320included in the AR device332. For example, after determining that an object334is not within a detection range of an AR device's sensors, the object detection module302may send a notification to a display module310that then removes object information from an augmented environment generated by the AR device332.

The data request module304may be configured to send a data request to an object334detected by the object detection module302. The data request may be for object data328and a digital certificate330that may be used to authenticate the object data328. In one example, the object data328may include metadata that describes the various attributes of the object334and may include context data for the object334that can be displayed in an augmented environment generated by the AR device332. In response to receiving the data request, the object334may send the object data328and the digital certificate330to the AR device332, and the object data328and the digital certificate330may be stored in a memory device322. The data request module304may be configured to send a notification to the data authentication module306requesting that the object data328stored in the memory device322be authenticated.

The data authentication module306may be configured to evaluate object data328to determine whether the object data328is valid (e.g., actually associated with an object334). The authentication methods described in association withFIG. 1may be utilized by the data authentication module306to authenticate object data328. For example, certificate authentication, attribute comparison authentication, reputation data authentication, and combinations thereof may be used to authenticate object data328provided by an object334. In one example, in response to a notification sent by the data request module304, the data authentication module306may be configured to retrieve a digital certificate330from a memory device322and validate the digital certificate330. In the case that the digital certificate330is successfully validated, the object data328may be considered valid. Additional authentication may be performed. For example, attributes described in the metadata included in the object data328may be compared to corresponding attributes identified in sensor data obtained using sensor devices320included in the AR device332. Also, reputation data associated with the object data328may be obtained from a community of object data reviewers and the reputation data may be evaluated to determine a general consensus whether the object data328is valid.

A decision to include the object data328in an augmented environment display may be made based in part on the results of the various authentications. For example, the data authentication module306may be configured to generate a validation score and assign the validation score to the object data328. The validation score may then be provided to a data filtering module308that may be used to identify object data328that may be displayed in an augmented environment based on authentication of the object data328and a user's preferences.

In one example, the data filtering module308may be configured to determine whether a validation score assigned to object data328satisfies a validation score threshold for displaying the object data328in an augmented environment generated by the AR device332. A user may control a level of authentication used by setting a validation score threshold to a value that matches the user's specifications. A user may also be provided with control over what types of objects334and what type of object data328may be displayed in an augmented environment display by setting object filtering parameters and data filtering parameters to the user's specifications. In the case that the validation score satisfies the validation score threshold and is in line with the user's filtering specifications, the display module310may be instructed to include the object data328, or a portion of the object data328in the augmented environment in respect to the object334displayed in the augmented environment.

As illustrated, the AR device332may include a display module310that may be used to render object data328and in some cases, objects334, on an AR device display. An AR device display may include optical projection systems, monitors, hand held devices, and display systems worn on the human body. An interactive graphical user interface312may be provided using the AR device display. An AR device332may include one or more processors314, memory devices322, and Input/Output (I/O) device communication316to enable communication between hardware devices and I/O components. Networking devices may be provided for communication across a network with remote computing devices, such as data service providers. The networking devices may provide wireless networking access. Examples of wireless network access may include cellular network access, WI-FI network access, or similar network access.

FIG. 4is a block diagram that illustrates an example system400in which an AR device414may be in network communication with a computing service environment402that hosts an object data authentication service406on one or more servers404. The object data authentication service406may be configured to authenticate and filter object data claiming to be associated with objects422encountered by the AR device414. Hosting the object data authentication service406in the computing service environment402may decrease the processing workload of an AR device414, and may provide benefits in cases where the AR device414may be configured with limited computing resources.

The object data authentication service406may include a data authentication module408and a data filtering module410that perform the functions described in association withFIG. 3. The AR device414may be configured to communicate with the object data authentication service406using a computing service interface420. For example, after detecting an object422and requesting object data from the object422, the AR device414may send a request to the object data authentication service406via the computing service interface420requesting that the object data be authenticated and filtered using a device profile412associated with the AR device414.

In an example where the object422sends authentication data and object data to the AR device414, the AR device414may forward the authentication data and object data to the object data authentication service406. In an example where the object422provides the AR device414with a reference (e.g., a URI) that may be used to retrieve the authentication data and object data, the AR device414may forward the reference to the object data authentication service406, which may then retrieve the authentication data and object data using the reference.

The computing service environment402may include computing resources for executing computing instances (e.g., virtual machines) as described in greater detail in association withFIG. 5. The various processes and/or other functionality contained within the computing service environment402may be executed on one or more processors that are in communication with one or more memory modules. The computing service environment402may include a number of computing devices that are arranged, for example, in one or more server banks or computer banks or other arrangements. The computing devices may support a computing environment using hypervisors, virtual machine monitors (VMMs) and other virtualization software.

A network416providing access to the computing service402may include any useful computing network, including an intranet, the Internet, a local area network, a wide area network, a wireless data network, or any other such network or combination thereof. Components utilized for such a system may depend at least in part upon the type of network and/or environment selected.

FIGS. 3 and 4illustrate that certain processing modules may be discussed in connection with this technology and these processing modules may be implemented as computing services. In one example configuration, a module may be considered a service with one or more processes executing on a server or other computer hardware. Such services may be centrally hosted functionality or a service application that may receive requests and provide output to other services or consumer devices. For example, modules providing services may be considered on-demand computing that are hosted in a server, virtualized service environment, grid or cluster computing system. An API may be provided for each module to enable a second module to send requests to and receive output from the first module. Such APIs may also allow third parties to interface with the module and make requests and receive output from the modules. WhileFIGS. 2, 3 and 4illustrate examples of systems that may implement the techniques above, many other similar or different environments are possible. The example environments discussed and illustrated above are merely representative and not limiting.

FIG. 5is a block diagram illustrating an example computing service500that may be used to execute and manage a number of computing instances504a-d. In particular, the computing service500depicted illustrates one environment in which the technology described herein may be used. The computing service500may be one type of environment that includes various virtualized service resources that may be used, for instance, to host computing instances504a-d.

The computing service500may be capable of delivery of computing, storage and networking capacity as a software service to a community of end recipients. In one example, the computing service500may be established for an organization by or on behalf of the organization. That is, the computing service500may offer a “private cloud environment.” In another example, the computing service500may support a multi-tenant environment, wherein a plurality of users may operate independently (i.e., a public cloud environment). Generally speaking, the computing service500may provide the following models: Infrastructure as a Service (“IaaS”), Platform as a Service (“PaaS”), and/or Software as a Service (“SaaS”). Other models may be provided. For the IaaS model, the computing service500may offer computers as physical or virtual machines and other resources. The virtual machines may be run as guests by a hypervisor, as described further below. The PaaS model delivers a computing platform that may include an operating system, programming language execution environment, database, and web server.

Application developers may develop and run their software solutions on the computing service platform without incurring the cost of buying and managing the underlying hardware and software. The SaaS model allows installation and operation of application software in the computing service500. End users may access the computing service500using networked client devices, such as desktop computers, laptops, tablets, smartphones, etc. running web browsers or other lightweight client applications, for example. Those familiar with the art will recognize that the computing service500may be described as a “cloud” environment.

The particularly illustrated computing service500may include a plurality of server computers502a-d. While four server computers are shown, any number may be used, and large data centers may include thousands of server computers. The computing service500may provide computing resources for executing computing instances504a-d. Computing instances504a-dmay, for example, be virtual machines. A virtual machine may be an instance of a software implementation of a machine (i.e. a computer) that executes applications like a physical machine. In the example of a virtual machine, each of the server computers502a-dmay be configured to execute an instance manager508a-dcapable of executing the instances. The instance manager508a-dmay be a hypervisor, virtual machine monitor (VMM), or another type of program configured to enable the execution of multiple computing instances504a-don a single server. Additionally, each of the computing instances504a-dmay be configured to execute one or more applications.

One or more server computers514and516may be reserved to execute software components for managing the operation of the computing service500and the computing instances504a-d. For example, a server computer514may execute an object data authentication service configured to authenticate and filter object data claiming to be associated with an object detected by an AR device.

A server computer516may execute a management component518. A user may access the management component518to configure various aspects of the operation of the computing instances504a-dpurchased by a user. For example, the user may setup computing instances504a-dand make changes to the configuration of the computing instances504a-d.

A deployment component522may be used to assist users in the deployment of computing instances504a-d. The deployment component522may have access to user account information524associated with the computing instances504a-d. The deployment component522may receive a configuration from a user that includes data describing how computing instances504a-dmay be configured. For example, the configuration may include an operating system, provide one or more applications to be installed in computing instances504a-d, provide scripts and/or other types of code to be executed for configuring computing instances504a-d, provide cache logic specifying how an application cache should be prepared, and other types of information. The deployment component522may utilize the user-provided configuration and cache logic to configure, prime, and launch computing instances504a-d. The configuration, cache logic, and other information may be specified by a user accessing the management component518or by providing this information directly to the deployment component522.

A network510may be utilized to interconnect the computing service500and the server computers502a-d,516. The network510may be a local area network (LAN) and may be connected to a Wide Area Network (WAN)512or the Internet, so that end users may access the computing service500. The network topology illustrated inFIG. 5has been simplified, many more networks and networking devices may be utilized to interconnect the various computing systems disclosed herein.

Moving now toFIG. 6, a flow diagram illustrates an example method600for evaluating an object and object data associated with the object. Starting in block602, an object may be detected by a sensor included in an AR device. Illustratively, the object may be detected using a network module (e.g., BLUETOOTH, NFC, or WI-FI), a camera, a microphone, or other sensor included in an AR device. The AR device may be within proximity of the object that allows the sensor to discover the presence of the object. The object may be configured with an interactive device that allows for communication with the AR device.

After detecting the object, as in block604the AR device may be configured to send a message to the object requesting object data associated with the object. In one example, the object itself may provide the requested object data. In another example, the object may be configured to provide a reference (e.g., URI) that can be used to retrieve the object data.

The object data may include metadata that describes the object. In particular, the object data may include metadata that describes an object type or object class (e.g., vehicles, structures, locations, people, or anything else that may be detectable by a sensor included in an AR device). As in block606, the object type of the object may be determined based in part on the metadata included in the object data. In one example, sensor data may also be used to determine the object type of the object.

As in block608, a determination may be made whether to accept the object data based in part on the type of the object. For example, an AR device user may set an object filtering parameter used to specify types of object data to display in an augmented environment display. As such, object data for object types not specified in the object filtering parameter may not be included in an augmented environment display. As a specific example, a user may add public transportation vehicles to an object filtering parameter, such that information associated with the public transportation vehicles (e.g., schedule, cost, available capacity, etc.) may be displayed in an augmented environment. Alternatively, a user may specify types of objects for which the user does not want to see object data (e.g., do not show information associated with public transportation vehicles).

In the case that the object data is associated with an object type that is not accepted for display in the augmented environment display generated by the AR device, the object data may be ignored, as in block610. In the case that the object data is associated with an acceptable object type, then as in block612, the object data may be authenticated using certificate authentication, attribute comparison authentication, and/or reputation data authentication as described earlier.

As in block614, if authentication of the object data fails, then as in block616, a message indicating the failure to authenticate the object data may be sent to the object. By sending the failure to authenticate message to the object, an owner of the object may be put on notice that authentication problems may be preventing object data from being displayed on AR devices. If authentication of the object data is successful, then as in block618, a selection of the object data may be displayed in the augmented environment generated by the AR device. For example, the object data may be displayed in the augmented environment according to an AR device user's preferences, such as displaying a summary level or detailed level of information for an object.

FIG. 7is a flow diagram illustrating an example method700for authenticating a provider of a data stream associated with an object702detected by an AR device704. The object702may be configured to provide a data stream to AR devices704that are within a communication range of the object. As one example, the object702may provide a virtual assistant programmed to provide information and instructions related to a geographical location (e.g., an airport, museum, or library) to AR device users located within proximity of the object702. As another example, the object702may include advertisement objects placed in public transportation channels (e.g., subways, busses, and taxis) that provide video advertisements to AR device users located within proximity of the object702.

As illustrated, an AR device704may detect710an object702by way of one or more sensors included in the AR device704. Having detected the object702, the AR device704may be configured to request pre-data712from the object702. The pre-data may be information that can be used to authenticate the identity of the object702, such that a data stream that claims to be associated with the object702can be trusted. For example, an AR device user may encounter an object702in an airport claiming that the object702provides flight information for the airport. However, the AR device user has no assurances that the object702provides valid flight information for the airport without first authenticating the identity of the object702.

Pre-data may include a digital certificate and object metadata, which may be returned714to the AR device704in response to the request712for the pre-data. The digital certificate may be authenticated716using certificate authentication. Sensor data generated by one or more sensors included in the AR device704may be collected718and the sensor data may be analyzed to identify object attributes represented in the sensor data. The object attributes represented in the sensor data may then be compared720to object attributes represented in the object metadata provided by the object702to determine whether the object attributes correspond with one another. As an example, an object702encountered at an airport may provide object metadata to an AR device704that includes location information for the object702. The location information may be compared to GPS coordinates obtained from a GPS module included in the AR device704to determine whether the location information corresponds to the GPS coordinates.

After authenticating the pre-data provided by the object702, the AR device704may request722a data stream from the object702. As an example, after authenticating an object702located at an airport, an AR device704may request a flight information stream that can be displayed in an augmented environment display generated by the AR device704. In response to the data stream request, the object702may provide724the data stream to the AR device704and the data stream may be displayed726in an augmented environment generated by the AR device704.

FIG. 8is a flow diagram that illustrates an example method800for evaluating object data associated with an object represented in an augmented environment. As in block810, an object located within proximity of a sensor included in a device may be detected. The device may be configured with a plurality of sensors that may be capable of detecting the presence of an object and collecting sensor data associated with the object. The device may also be configured to display object data for the object in an augmented environment display.

After detecting the object, as in block820, object data associated with the object may be requested for display within the augmented environment display generated by the device. In one example, the object may provide the object data to the device. In another example, the object data may be obtained from a data service provider identified by the object.

After the object data has been obtained, the object data may be authenticated. A combination of authentication methods may be used. As in block830, the object data may be authenticated using an authenticating authority that certifies that the object data is valid. For example, a digital certificate issued by a certificate authority may be provided to the device and the digital certificate may be authenticated using a certificate authentication technique.

As in block840, the object data may then be authenticated by determining that an object attribute represented by the object data corresponds to the object attribute represented by sensor data obtained from the sensor. After the object data has been authenticated, as in block850, at least a portion of the object data may be displayed in the augmented environment display generated by the device.

FIG. 9illustrates a computing device910on which modules of this technology may execute. A computing device910is illustrated on which a high level example of the technology may be executed. The computing device910may include one or more processors912that are in communication with memory devices920. The computing device910may include a local communication interface918for the components in the computing device. For example, the local communication interface918may be a local data bus and/or any related address or control busses as may be desired.

The memory device920may contain modules924that are executable by the processor(s)912and data for the modules924. For example, the memory device920may include an object detection module, data request module, data authentication module, data filtering module, and other modules. The modules924may execute the functions described earlier. A data store922may also be located in the memory device920for storing data related to the modules924and other applications along with an operating system that is executable by the processor(s)912.

Other applications may also be stored in the memory device920and may be executable by the processor(s)912. Components or modules discussed in this description that may be implemented in the form of software using high programming level languages that are compiled, interpreted or executed using a hybrid of the methods.

The computing device may also have access to I/O (input/output) devices914that are usable by the computing devices. An example of an I/O device is a display screen930that is available to display output (an augmented environment) from the computing devices. Networking devices916and similar communication devices may be included in the computing device. The networking devices916may be wired or wireless networking devices that connect to the internet, a LAN, WAN, or other computing network.

The components or modules that are shown as being stored in the memory device920may be executed by the processor(s)912. The term “executable” may mean a program file that is in a form that may be executed by a processor912. For example, a program in a higher level language may be compiled into machine code in a format that may be loaded into a random access portion of the memory device920and executed by the processor912, or source code may be loaded by another executable program and interpreted to generate instructions in a random access portion of the memory to be executed by a processor. The executable program may be stored in any portion or component of the memory device920. For example, the memory device920may be random access memory (RAM), read only memory (ROM), flash memory, a solid state drive, memory card, a hard drive, optical disk, floppy disk, magnetic tape, or any other memory components.

The processor912may represent multiple processors and the memory device920may represent multiple memory units that operate in parallel to the processing circuits. This may provide parallel processing channels for the processes and data in the system. The local interface918may be used as a network to facilitate communication between any of the multiple processors and multiple memories. The local interface918may use additional systems designed for coordinating communication such as load balancing, bulk data transfer and similar systems.