VALIDATING AND FILTERING MIXED REALITY CONTENT

In an approach to validating and filtering mixed reality content, one or more computer processors receive a request for mixed reality content from a user. One or more computer processors retrieve the mixed reality content. One or more computer processors determine a mobility of the user in a physical environment. One or more computer processors determine the mixed reality content does not comply with at least one pre-defined policy associated with a physical location boundary within the physical environment. One or more computer processors filter the mixed reality content to comply with the at least one pre-defined policy. Based on the mobility of the user in the physical environment, one or more computer processors process the filtered mixed reality content. One or more computer processors display the filtered mixed reality content.

BACKGROUND OF THE INVENTION

The present invention relates generally to the field of mixed reality, and more particularly to validating and filtering mixed reality content.

Mixed reality (MR) is a term used to describe the merging of a real-world environment and a computer-generated one. Physical and virtual objects may co-exist in mixed reality environments and interact in real time. MR technology enables real and virtual elements to interact with one another and enables the user to interact with virtual elements as they would in the real-world. MR may not be considered a fully immersive experience since it maintains a connection to the real world. In a mixed reality environment, wherever a user focuses attention, while wearing MR technology, the 3D content that the user encounters in the space reacts to the user as it would in the real-world. For example, a virtual object will move closer to the user as the user moves closer to it, and the user can interact with the object, e.g., the user can turn an object using gestures.

Edge computing is a recent trend that extends cloud computing and the Internet of Things (IoT) to the edge of the network, bringing more computational power and resources closer to end users by increasing the number of endpoints and positioning them nearer to consumers, be they users or devices. Edge computing is a distributed computing paradigm that brings computation and data storage closer to the sources of data. This is expected to improve response times and save bandwidth. Edge computing is an architecture rather than a specific technology, and a topology- and location-sensitive form of distributed computing. Edge computing architectures are based on existing technologies and distributed systems paradigms, providing a range of well-understood components to create the most effective architectures for delivering edge use cases.

In any MR interaction taking place in a physical environment, such as an office building or activity floor, a user can interact with virtual reality (VR) objects. However, if these objects contain malicious content or misleading information, or the MR system is hacked, the user may be misled into performing an incorrect action. To prevent this, it is necessary to develop a method and system for validating MR content according to the policy of the physical environment at the edge and for IoT environments.

SUMMARY

A first aspect of the present invention discloses a computer-implemented method including one or more computer processors receiving a request for mixed reality content from a user. One or more computer processors retrieve the mixed reality content. One or more computer processors determine a mobility of the user in a physical environment. One or more computer processors determine the mixed reality content does not comply with at least one pre-defined policy associated with a physical location boundary within the physical environment. One or more computer processors filter the mixed reality content to comply with the at least one pre-defined policy. Based on the mobility of the user in the physical environment, one or more computer processors process the filtered mixed reality content. One or more computer processors display the filtered mixed reality content. The present invention has the advantage of acting as a security layer and ensuring the mixed reality content is in compliance with security policies.

A second aspect of the present invention discloses a computer program product including one or more computer readable storage media and program instructions to receive a request for mixed reality content from a user. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media, to retrieve the mixed reality content. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media, to determine a mobility of the user in a physical environment. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media, to determine the mixed reality content does not comply with at least one pre-defined policy associated with a physical location boundary within the physical environment. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media, to filter the mixed reality content to comply with the at least one pre-defined policy. Based on the mobility of the user in the physical environment, the program instructions include program instructions, stored on at least one of the one or more computer-readable storage media, to process the filtered mixed reality content. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media, to display the filtered mixed reality content.

A third aspect of the present invention discloses a computer system including one or more computer processors one or more computer-readable memories and one or more computer readable storage media and program instructions, stored on at least one of the one or more computer readable storage media for execution by at least one of the one or more computer processors via at least one of the one or more memories, to receive a request for mixed reality content from a user. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media, to retrieve the mixed reality content. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media for execution by at least one of the one or more computer processors via at least one of the one or more memories to determine a mobility of the user in a physical environment. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media for execution by at least one of the one or more computer processors via at least one of the one or more memories to determine the mixed reality content does not comply with at least one pre-defined policy associated with a physical location boundary within the physical environment. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media for execution by at least one of the one or more computer processors via at least one of the one or more memories to filter the mixed reality content to comply with the at least one pre-defined policy. Based on the mobility of the user in the physical environment, the program instructions include program instructions, stored on at least one of the one or more computer-readable storage media for execution by at least one of the one or more computer processors via at least one of the one or more memories to process the filtered mixed reality content. The program instructions include program instructions, stored on at least one of the one or more computer-readable storage media for execution by at least one of the one or more computer processors via at least one of the one or more memories to display the filtered mixed reality content.

In another aspect, the present invention discloses where the mobility of the user in the physical environment includes at least one of a location coordinate, a relative position of the user, and whether the user is moving from one physical location to another. An advantage of tracking the mobility of the user in the physical environment enable is that it enables edge devices in the environment to collaborate with each other regarding which instance of the present invention maintains control of the mixed reality content validation.

DETAILED DESCRIPTION

Embodiments of the present invention recognize that computing security may be improved by providing a mixed reality (MR) control system on a plurality of edge devices within a physical environment that connect with an MR device and validate MR content prior to displaying the content to prevent malicious content and/or misleading information from being presented to a user. Embodiments of the present invention also recognize that additional computing security may be gained by filtering the MR content based on one or more pre-defined policies associated with each edge device in the physical environment prior to projecting the content for the user. Additionally, embodiments of the present invention recognize that efficiency may be gained by adapting the MR content to the location and mobility of the user in the space. By utilizing edge computing among edge devices, one skilled in the art would recognize that the bandwidth associated with utilizing a centralized server or a cloud computing system is reduced. In addition, the security of information obtained by the edge devices is improved because unnecessary or inappropriate information can be filtered near the source. Implementation of embodiments of the invention may take a variety of forms, and exemplary implementation details are discussed subsequently with reference to the Figures.

Distributed data processing environment100includes edge device104, edge device112, MR device114, and server computer118, all interconnected over network102. In an embodiment, distributed data processing environment100represents an edge computing ecosystem in a physical environment. Network102can be, for example, a telecommunications network, a local area network (LAN), a wide area network (WAN), such as the Internet, or a combination of the three, and can include wired, wireless, or fiber optic connections. Network102can include one or more wired and/or wireless networks capable of receiving and transmitting data, voice, and/or video signals, including multimedia signals that include voice, data, and video information. In general, network102can be any combination of connections and protocols that will support communications between edge device104, edge device112, MR device114, server computer118, and other computing devices (not shown) within distributed data processing environment100. Distributed data processing environment100may be implemented in computing environment300, shown inFIG.3.

Edge device104and edge device112represent an array of edge devices located within a physical environment and can each be one or more of a laptop computer, a tablet computer, a smart phone, smart watch, a smart speaker, manufacturing equipment, or any programmable electronic device capable of communicating with various components and other computing devices (not shown) within distributed data processing environment100, via a network, such as network102. In general, edge device104and edge device112each represents one or more programmable electronic devices or combination of programmable electronic devices capable of executing machine readable program instructions. Edge device104and edge device112can each be static or mobile. For example, edge device104and edge device112may each be a mobile device, such as a camera, or a static device, such as a network switch. Edge device104and edge device112may each have different functions, for example, security, end-user applications, and operating machinery. Edge device104and edge device112each have a corresponding physical location boundary, i.e., defined, 3D coordinates, in which the edge device is active and responsible for controlling the MR content. Edge device104includes MR content control system1061and edge device policy database1101. Edge device112includes MR content control system106Nand edge device policy database110N. Edge device104and edge device112may each include internal and external hardware components, as depicted and described in further detail with respect to computer301ofFIG.3.

MR content control system1061and MR content control system106N. herein collectively referred to as MR content control system106, represent a mixed reality control system resident on a plurality of edge devices. As used herein. N represents a positive integer, and accordingly the number of scenarios implemented in a given embodiment of the present invention is not limited to those depicted inFIG.1. MR content control system106validates and, if necessary, filters MR content prior to displaying the content in order to prevent inappropriate, malicious, and/or misleading content from being displayed to a user. MR content control system106receives a request for MR content. MR content control system106retrieves the MR content. MR content control system106determines the user mobility in the physical environment. MR content control system106analyzes the MR context. MR content control system106determines whether the MR content complies with a pre-defined policy, and, if not, then MR content control system106filters the MR content. MR content control system106processes the MR content to the user mobility. MR content control system106displays the MR content. MR content control system1061includes context analysis module1081. MR content control system106Nincludes context analysis module108N. MR content control system106is depicted and described in further detail with respect toFIG.2.

Context analysis module1081and context analysis module108N, herein collectively referred to as context analysis module108, represent a component of MR content control system106that analyzes the context of the physical environment in which the user resides when the user requests MR content. As used herein, N represents a positive integer, and accordingly the number of scenarios implemented in a given embodiment of the present invention is not limited to those depicted inFIG.1. In the depicted embodiment, context analysis module108is a standalone component of MR content control system106. In another embodiment, the function of context analysis module108is integrated into MR content control system106.

Edge device policy database1101and edge device policy database110N, herein collectively referred to as edge device policy database110, stores one or more policies associated with each edge device in distributed data processing environment100, such as edge device104and edge device112. As used herein, N represents a positive integer, and accordingly the number of scenarios implemented in a given embodiment of the present invention is not limited to those depicted inFIG.1. In the depicted embodiment, edge device policy database110resides on each edge device in distributed data processing environment100. In another embodiment, edge device policy database110may reside elsewhere within distributed data processing environment100, provided that MR content control system106has access to edge device policy database110, via network102. A database is an organized collection of data. Edge device policy database110can be implemented with any type of storage device capable of storing data and configuration files that can be accessed and utilized by MR content control system106such as a database server, a hard disk drive, or a flash memory. Edge device policy database110stores information used by and generated by MR content control system106. For example, edge device policy database110stores results generated by context analysis module108when filtering and/or validating MR content. Edge device policy database110also stores 3D coordinates associated with a physical location boundary corresponding to each edge device. Edge device policy database110also stores one or more policies associated with the corresponding edge device and/or the physical location boundary associated with the edge device. For example, if the physical environment is a manufacturing facility, then a policy may be authorization to access the facility and/or certain equipment, such as a table saw, within the facility or within the physical location boundary associated with the corresponding edge device. Edge device policy database110also stores a user profile of the user of MR device114. The user profile includes data associated with the user, including, but not limited to, the name of the user, an address, an email address, an image of the user, a voice sample, a phone number, a credit card number, an account number, a business loyalty account number, a student identification number, an employer, a job role, a job family, a business unit association, a job seniority, a job level, a resume, a social network affiliation, a current geographic location, an access credential for MR content, etc.

It should be noted herein that in the described embodiments, participating parties have consented to being recorded and monitored, and participating parties are aware of the potential that such recording and monitoring may be taking place. In various embodiments, for example, when downloading or operating an embodiment of the present invention, the embodiment of the invention presents a terms and conditions prompt enabling the user to opt-in or opt-out of participation. Similarly, in various embodiments, emails and texts begin with a written notification that the user's information may be recorded or monitored and may be saved, for the purpose of validating and filtering mixed reality content. These embodiments may also include periodic reminders of such recording and monitoring throughout the course of any such use. Certain embodiments may also include regular (e.g., daily, weekly, monthly) reminders to the participating parties that they have consented to being recorded and monitored for validating and filtering mixed reality content and may provide the participating parties with the opportunity to opt-out of such recording and monitoring if desired. Furthermore, to the extent that any non-participating parties' actions are monitored (for example, when outside vehicles are viewed), such monitoring takes place for the limited purpose of providing navigation assistance to a participating party, with protections in place to prevent the unauthorized use or disclosure of any data for which an individual might have a certain expectation of privacy.

MR device114can be one or more of a laptop computer, a tablet computer, a smart phone, smart watch, a smart speaker, or any programmable electronic device capable of communicating with various components and devices within distributed data processing environment100, via network102. MR device114may be a wearable computer. Wearable computers are miniature electronic devices that may be worn by the bearer under, with, or on top of clothing, as well as in or connected to glasses, hats, or other accessories. Wearable computers are especially useful for applications that require more complex computational support than merely hardware coded logics. In an embodiment, the wearable computer may be in the form of a smart watch. In one embodiment, the wearable computer may be in the form of a head mounted display (HMD). The HMD may take the form-factor of a pair of glasses, such as augmented reality (AR) glasses, which is a device for viewing mixed reality and/or augmented reality scenarios. In the embodiment where the HMD is a pair of AR glasses, the AR glasses can capture eye gaze information from a gaze point tracker, such as a camera associated with MR device114. In general, MR device114represents one or more programmable electronic devices or combination of programmable electronic devices capable of executing machine readable program instructions and communicating with other computing devices (not shown) within distributed data processing environment100via a network, such as network102. MR device114includes an instance of MR user interface116.

MR user interface116provides an interface between the user of MR device114and edge device104, edge device112, and server computer118. In one embodiment, MR user interface116is mobile application software. Mobile application software, or an “app,” is a computer program designed to run on smart phones, tablet computers and other mobile devices. In one embodiment, MR user interface116may be a graphical user interface (GUI) or a web user interface (WUI) and can display text, documents, web browser windows, user options, application interfaces, and instructions for operation, and include the information (such as graphic, text, and sound) that a program presents to a user and the control sequences the user employs to control the program. In an embodiment, MR user interface116enables a user of MR device114to request MR content from MR content database120. In an embodiment, MR user interface116enables a user to store edge device specific policies in edge device policy database110. In an embodiment, MR user interface116also enables the user of MR device114to store data associated with a user profile. In an embodiment, MR user interface116enables the user of MR device114to define physical boundaries associated with each edge device in the physical environment.

Server computer118can be a standalone computing device, a management server, a web server, a mobile computing device, or any other electronic device or computing system capable of receiving, sending, and processing data. In other embodiments, server computer118can represent a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment. In another embodiment, server computer118can be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smart phone, an edge device, a containerized workload, or any programmable electronic device capable of communicating with edge device104, edge device112, MR device114, and other computing devices (not shown) within distributed data processing environment100via network102. In another embodiment, server computer118represents a computing system utilizing clustered computers and components (e.g., database server computers, application server computers, etc.) that act as a single pool of seamless resources when accessed within distributed data processing environment100. Server computer118includes MR content database120. Server computer118may include internal and external hardware components, as depicted and described in further detail with respect to computer301ofFIG.3.

In the depicted embodiment, MR content database120resides on server computer118. In another embodiment, MR content database120may reside elsewhere within distributed data processing environment100, provided that MR content control system106and MR device114have access to MR content database120, via network102. MR content database120can be implemented with any type of storage device capable of storing data and configuration files that can be accessed and utilized by MR content control system106and MR device114such as a database server, a hard disk drive, or a flash memory. MR content database120stores MR content for display on MR device114. For example, the MR content may provide guidance to a user regarding how to navigate from one location to another in a manufacturing facility in order to complete a series of tasks.

The present invention may contain various accessible data sources, such as edge device policy database110and MR content database120, that may include personal data, content, or information the user wishes not to be processed. Personal data includes personally identifying information or sensitive personal information as well as user information, such as tracking or geolocation information. Processing refers to any operation, automated or unautomated, or set of operations such as collecting, recording, organizing, structuring, storing, adapting, altering, retrieving, consulting, using, disclosing by transmission, dissemination, or otherwise making available, combining, restricting, erasing, or destroying personal data. MR content control system106enables the authorized and secure processing of personal data. MR content control system106provides informed consent, with notice of the collection of personal data, allowing the user to opt in or opt out of processing personal data. Consent can take several forms. Opt-in consent can impose on the user to take an affirmative action before personal data is processed. Alternatively, opt-out consent can impose on the user to take an affirmative action to prevent the processing of personal data before personal data is processed. MR content control system106provides information regarding personal data and the nature (e.g., type, scope, purpose, duration, etc.) of the processing. MR content control system106provides the user with copies of stored personal data. MR content control system106allows the correction or completion of incorrect or incomplete personal data. MR content control system106allows the immediate deletion of personal data.

FIG.2is a flowchart depicting operational steps of MR content control system106, on edge device104and/or edge device112within distributed data processing environment100ofFIG.1, for validating and filtering mixed reality content, in accordance with an embodiment of the present invention.

MR content control system106receives a request for MR content (step202). In an embodiment, when a user of MR device114begins an MR interaction activity, MR content control system106receives a request for MR content. In an embodiment, when the user of MR device114requests MR content stored in MR content database120, via MR user interface116, MR content control system106receives the request. In an embodiment, MR content control system106receives the request directly from MR user interface116. In another embodiment, the request passed to MR content database120triggers MR content control system106to determine the user has requested MR content. In an embodiment, MR content control system106also receives input from MR device114. For example, input from MR device114may include the visual surrounding of the user, the eye gaze of the user, etc.

MR content control system106retrieves the MR content (step204). In an embodiment, MR content control system106retrieves the requested MR content from MR content database120. In another embodiment, in response to the request from the user, MR user interface116communicates the request to MR content database120, MR content database120transmits the requested content to MR content control system106, and MR content control system106receives the requested content.

MR content control system106determines the user mobility in the physical environment (step206). In an embodiment, MR content control system106tracks the user's interaction in the physical environment and determines the location coordinates and/or relative position of the user and whether the user is moving from one physical location to another. For example, in an embodiment where MR device114includes a global positioning service (GPS), MR content control system106determines the GPS coordinates of the user as the user moves through the environment. An advantage of tracking the mobility of the user in the environment is that, based on the physical location boundary of each edge device, MR content control system106can determine on which edge device to process the requested MR content corresponding to the location of the user. Another advantage of determining the user mobility in the physical environment is to enable edge device104and edge device112(and any other edge devices in the environment, not shown) to collaborate with each other regarding which instance of MR content control system1061-x executes the MR content control process.

MR content control system106analyzes the context of the environment (step208). In an embodiment, MR content control system106analyzes the context of the physical environment to determine contextual attributes of the environment. For example, MR content control system106may identify a background color, one or more participants in the MR activity, a time of day, etc. In an embodiment, context analysis module108analyzes the context of the environment. In an embodiment, MR content control system106analyzes the context of the user in the environment. For example, MR content control system106determines if the user is an employee that works in the environment. An advantage of analyzing the context of the environment is that while MR content control system106, as will be discussed below, filters the MR content based on a pre-defined policy, due to the surrounding environment, the MR content may need additional filter.

MR content control system106determines whether the MR content complies with a pre-defined policy (decision block210). In an embodiment, based on the context of the user in the physical environment, MR content control system106validates that the requested MR content complies with a pre-defined policy associated with the edge device physical location boundary in which the user currently resides. For example, if the requested MR content includes financial information, the pre-defined policy may be to confirm the user is authorized to view the financial information. In another example, if the requested MR content includes personal information, such as names, the pre-defined policy may be to redact names and replace them with job roles if the user is not authorized to view the names. In an embodiment, each instance of MR content control system106in each edge device in the physical environment, simultaneously determines whether the requested MR content complies with each corresponding pre-defined policy.

If MR content control system106determines the MR content does not comply with the pre-defined policy (“no” branch, decision block210), then MR content control system106filters the MR content (step212). In an embodiment, MR content control system106filters the requested MR content such that, after being filtered, the content is appropriate and complies with the corresponding pre-defined policy of each edge device. Continuing the previous examples, if the requested MR content includes financial information, and MR content control system106determines the user is not authorized to view the financial information, then MR content control system106filters out the financial information. Or, if the requested MR content includes personal information, and MR content control system106determines the user is not authorized to view the personal information then MR content control system106replaces the personal information with generic information. In another example, if MR content control system106determines the requested MR content includes malicious and/or misleading content or any content that contains a cyber threat, then MR content control system106removes the malicious and/or misleading content. The advantage of filtering the MR content is that MR content control system106acts as a security layer and ensures the MR content is in compliance with security policies.

Responsive to filtering the content, or if MR content control system106determines the MR content complies with the pre-defined policy (“yes” branch, decision block210), then MR content control system106processes the MR content to the user mobility (step214). In an embodiment, once MR content control system106validates the MR content is in compliance with one or more pre-defined policies associated with the edge device in the physical location in which the user resides, MR content control system106processes the MR content to adapt it to various mobility parameters of the user and align the content with the physical environment. For example, if MR content control system106determines the user is moving from the physical location boundary of edge device104toward the physical location boundary of edge device112, then MR content control system106adapts the MR content from complying with the pre-defined policy of edge device104to complying with the pre-defined policy of edge device112.

In an embodiment, MR content control system106leverages intention analysis to identify the intent of the user with respect to interaction with the MR content when processing the MR content for the user. In an embodiment, MR content control system106uses historical data associated with how the user, or similar users, have interacted with specific MR content in the past when processing the MR content for the user.

MR content control system106displays the MR content (step216). In an embodiment, MR content control system106displays the validated MR content in MR device114, via MR user interface116. In another embodiment, MR content control system106stores the validated MR content in MR content database120and instructs MR device114to retrieve the validated content for display to the user.

FIG.3is an example diagram of a distributed data processing environment in which aspects of one or more of the illustrative embodiments may be implemented, and at least some of the computer code involved in performing the inventive methods may be executed, in accordance with an embodiment of the present invention, in accordance with an embodiment of the present invention. It should be appreciated thatFIG.3provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments can be implemented. Many modifications to the depicted environment can be made.

Computing environment300contains an example of an environment for the execution of at least some of the computer code involved in performing the inventive methods, such as MR content control system106for validating and filtering MR content. In addition to MR content control system106, computing environment300includes, for example, computer301, wide area network (WAN)302, end user device (EUD)303, remote server304, public cloud305, and private cloud306. In this embodiment, computer301includes processor set310(including processing circuitry320and cache321), communication fabric311, volatile memory312, persistent storage313(including operating system322and MR content control system106, as identified above), peripheral device set314(including user interface (UI), device set323, storage324, and Internet of Things (IoT) sensor set325), and network module315. Remote server304includes remote database330. Public cloud305includes gateway340, cloud orchestration module341, host physical machine set342, virtual machine set343, and container set344.

Processor set310includes one, or more, computer processors of any type now known or to be developed in the future. Processing circuitry320may be distributed over multiple packages, for example, multiple, coordinated integrated circuit chips. Processing circuitry320may implement multiple processor threads and/or multiple processor cores. Cache321is memory that is located in the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores running on processor set310. Cache memories are typically organized into multiple levels depending upon relative proximity to the processing circuitry. Alternatively, some, or all, of the cache for the processor set may be located “off chip.” In some computing environments, processor set310may be designed for working with qubits and performing quantum computing.

Computer readable program instructions are typically loaded onto computer301to cause a series of operational steps to be performed by processor set310of computer301and thereby effect a computer-implemented method, such that the instructions thus executed will instantiate the methods specified in flowcharts and/or narrative descriptions of computer-implemented methods included in this document (collectively referred to as “the inventive methods”). These computer readable program instructions are stored in various types of computer readable storage media, such as cache321and the other storage media discussed below. The program instructions, and associated data, are accessed by processor set310to control and direct performance of the inventive methods. In computing environment300, at least some of the instructions for performing the inventive methods may be stored in MR content control system106in persistent storage313.

Volatile memory312is any type of volatile memory now known or to be developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, the volatile memory is characterized by random access, but this is not required unless affirmatively indicated. In computer301, the volatile memory312is located in a single package and is internal to computer301, but, alternatively or additionally, the volatile memory may be distributed over multiple packages and/or located externally with respect to computer301.

Persistent storage313is any form of non-volatile storage for computers that is now known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is being supplied to computer301and/or directly to persistent storage313. Persistent storage313may be a read only memory (ROM), but typically at least a portion of the persistent storage allows writing of data, deletion of data and re-writing of data. Some familiar forms of persistent storage include magnetic disks and solid-state storage devices. Operating system322may take several forms, such as various known proprietary operating systems or open-source Portable Operating System Interface type operating systems that employ a kernel. The code included in MR content control system106typically includes at least some of the computer code involved in performing the inventive methods.

Peripheral device set314includes the set of peripheral devices of computer301. Data communication connections between the peripheral devices and the other components of computer301may be implemented in various ways, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made by cables (such as universal serial bus (USB) type cables), insertion type connections (for example, secure digital (SD) card), connections made through local area communication networks and even connections made through wide area networks such as the internet. In various embodiments, UI device set323may include components such as a display screen, speaker, microphone, wearable devices (such as goggles and smart watches), keyboard, mouse, printer, touchpad, game controllers, and haptic devices. Storage324is external storage, such as an external hard drive, or insertable storage, such as an SD card. Storage324may be persistent and/or volatile. In some embodiments, storage324may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer301is required to have a large amount of storage (for example, where computer301locally stores and manages a large database) then this storage may be provided by peripheral storage devices designed for storing very large amounts of data, such as a storage area network (SAN) that is shared by multiple, geographically distributed computers. IoT sensor set325is made up of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

End user device (EUD)303is any computer system that is used and controlled by an end user (for example, a customer of an enterprise that operates computer301) and may take any of the forms discussed above in connection with computer301. EUD303typically receives helpful and useful data from the operations of computer301. For example, in a hypothetical case where computer301is designed to provide a recommendation to an end user, this recommendation would typically be communicated from network module315of computer301through WAN302to EUD303. In this way. EUD303can display, or otherwise present, the recommendation to an end user. In some embodiments, EUD303may be a client device, such as thin client, heavy client, mainframe computer, desktop computer and so on.

Remote server304is any computer system that serves at least some data and/or functionality to computer301. Remote server304may be controlled and used by the same entity that operates computer301. Remote server304represents the machine(s) that collect and store helpful and useful data for use by other computers, such as computer301. For example, in a hypothetical case where computer301is designed and programmed to provide a recommendation based on historical data, then this historical data may be provided to computer301from remote database330of remote server304.

Public cloud305is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and/or other computer capabilities, especially data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages sharing of resources to achieve coherence and economies of scale. The direct and active management of the computing resources of public cloud305is performed by the computer hardware and/or software of cloud orchestration module341. The computing resources provided by public cloud305are typically implemented by virtual computing environments that run on various computers making up the computers of host physical machine set342, which is the universe of physical computers in and/or available to public cloud305. The virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set343and/or containers from container set344. It is understood that these VCEs may be stored as images and may be transferred among and between the various physical machine hosts, either as images or after instantiation of the VCE. Cloud orchestration module341manages the transfer and storage of images, deploys new instantiations of VCEs and manages active instantiations of VCE deployments. Gateway340is the collection of computer software, hardware, and firmware that allows public cloud305to communicate through WAN302.

Private cloud306is similar to public cloud305, except that the computing resources are only available for use by a single enterprise. While private cloud306is depicted as being in communication with WAN302, in other embodiments a private cloud may be disconnected from the internet entirely and only accessible through a local/private network. A hybrid cloud is a composition of multiple clouds of different types (for example, private, community or public cloud types), often respectively implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technology that enables orchestration, management, and/or data/application portability between the multiple constituent clouds. In this embodiment, public cloud305and private cloud306are both part of a larger hybrid cloud.

The foregoing descriptions of the various embodiments of the present invention have been presented for purposes of illustration and example but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The terminology used herein was chosen to best explain the principles of the embodiment, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.