System and method of creating, hosting, and accessing virtual reality projects

A system and a method of creating, hosting, and accessing a VR project is described. Media files are imported into a software application running on a system and details of interaction between the media files are provided. The media files are processed based on the details of interaction for creation of the VR project. The VR project may include multiple versions of media files prepared based on configurations of user devices requiring access to the VR project. A user device may place a request to a server hosting the VR project. The server may identify an optimal version of media files for the user device, based on configurations of the user device. The optimal version of the media files along with a corresponding VRXF file including details of interactions between the media files is transmitted to the user device, for delivering a best VR experience of the VR project.

FIELD OF INVENTION

The present invention is directed to Virtual Reality (VR), and more particularly to the distributed management of VR projects.

BACKGROUND OF INVENTION

Technology has advanced to a level where different techniques, platforms, and high end graphic processing devices are available for processing information, specifically media files, for generation of Virtual Reality (VR) data.

Typically, very high quality 360-degree media files are processed for development of VR data. The VR data is generally developed for being played on supporting VR devices, i.e., VR devices including specifications that are compatible with and sufficient to process the VR data. Sometimes, the VR data is made available in different formats/versions compatible with different VR devices. In such cases, the VR devices do not possess the capability to identify the optimal VR data from several formats/versions of the VR data.

Further, programmers are required to write massive program codes defining the instructions for the processing of media files. Based on these instructions, the media files are processed to generate VR data. Development of these program codes requires a vast knowledge of 3D programming and processing of media files.

Therefore, there exists a need for a simple and convenient manner of generation of VR projects without requiring knowledge of programming and 3D processing of media files, and identification of different VR projects optimal for being played on any VR device.

SUMMARY

It is one of the objectives of the present disclosure to provide a method of identifying and executing an optimal version of media files of a Virtual Reality (VR) project, and that avoids the drawbacks of the prior art.

In order to achieve the objectives, the present disclosure provides a technical feature in which a system and a method of creating, hosting, and accessing a VR project is described. Media files may be imported into a software application running on a user device, by a VR project creator. The VR project creator may also provide creative inputs such as interaction between the media files. Details of interaction between the media files may be provided to the software application. The software application may process the media files based on the details of the interaction, for creation of a VR project. A user device may place a request to a server hosting multiple versions of media files related to the VR project, for accessing the VR project. The server may identify an optimal version of media files of the corresponding VR project from the multiple versions of media files, based on the user device's configuration, such as its hardware capabilities (processing, display, storage, and communication capability) and firmware type and version. The user device may play the optimal version of the media files of the VR project.

In one implementation, a system for creation and management of VR projects may comprise a memory and a processor operatively coupled to the memory. The memory may be configured to store programmed instructions. The processor may be configured to execute the programmed instructions to allow a VR project creator to select a plurality of media files and define interactions between the plurality of media files. The processor may be further configured to create a VR project including a text file and the plurality of media files. The text file includes references to storage locations of the plurality of media files and details of the interactions between the plurality of media files. The processor may be further configured to manage storage of the text file and provide access of the text file to a user device in response to a user request.

In one aspect, the text file is a VRXF file.

In one aspect, the plurality of media files is stored in the memory or is available over Internet.

In one aspect, the processor executes one or more applications for editing the plurality of media files.

In one aspect, the processor executes generative data models for generating a VRXF file and one or more media files to be used for creating the VR project.

In one aspect, the text file is executed by a VR playing component running on the user device.

In one aspect, the storage of the text file is performed, or the access of the text file is provided based on rules set by a third party.

In one aspect, the processor processes the plurality of media files to generate multiple versions of the plurality of media files suitable for different configurations of different user devices, wherein the processing includes modifying resolution, modifying orientation, and compression of the plurality of media files.

In one aspect, the processor provides access of an optimal version of the plurality of media files to the user device, and wherein the processor determines the optimal version of the plurality of media files based on configuration details of the user device, received through the user request.

In one aspect, the configuration details include software type and version, hardware capability, and Degrees of Freedom (DoF) available for interaction.

In one aspect, the creation of the VR project is performed by a first processing device and the storage and access of the text file is managed by a second processing device.

In one implementation, a method of creation and management of VR projects is described. The method comprises allowing a VR project creator to select a plurality of media files and define interactions between the plurality of media files. The method further comprises creating a VR project including a text file and the plurality of media files. The text file includes references to storage locations of the plurality of media files and details of the interactions between the plurality of media files. The method further comprises managing storage of the text file and providing access of the text file to a user device in response to a user request.

DETAILED DESCRIPTION OF THE EMBODIMENTS

In one working embodiment of the present disclosure, referring toFIG.1, distributed management of Virtual Reality (VR) projects is explained.FIG.1illustrates an example illustration of an environment for distributed management of VR projects. A system102operable by a VR project creator104may be connected with a server106. The system102may be connected with the server106through a communication network108. To create a VR project, the VR project creator104may access a software application running on the system102or the server106. The VR project creator104may access the software application by operating one of several input means110including a touch-controlled device, a mouse, or a handheld controller.

The software application may receive one or more media files or one or more media assets from the VR project creator104. The one or more media files may be present in a local storage112or a remote storage114. The local storage112may be a memory of the system102or a memory of data storage/processing device connected with the system102through a wired or wireless connection. The remote storage114may correspond to a web location present over Internet or over a private network or a memory120of the server106.

Although in preferred implementations, the one or more media files may include 3600 and 2D image files, it may be possible to utilize other forms of media files such as 360° and 2D video files, audio files, 3D model files, 2D images, 2D videos, and livestream media files in different implementations.

After providing the one or more media files, the VR project creator104may define interaction between the one or more media files, for example sequence, position, orientation, distance, and behavior of the one or more media files. The VR project creator104may define the interactions between the one or more media files by operating one of the several input means110. The VR project creator104may define the interactions by performing simple actions, such as dragging and dropping the one or more media files, and zooming in and out of a scene of a media file through pinching action.

In an example and not by way of any limitation, the VR project creator104may establish a connection between one or more scenes. Further, the VR project creator104may add an image or an audio file to the scene. The VR project creator104may add actions such as show or hide objects, activate or deactivate objects, start, pause or stop animation, loop or unloop video sound, start or pause video sound, open Uniform Resource Locator (URL), and send webhook.

The server106may include a processor118and the memory120. The memory120may be configured to store programmed instructions. The processor118may be configured to execute the programmed instructions to implement different functions described henceforth with reference toFIG.3throughFIG.5.

Referring now toFIG.2illustrating an example User Interface (UI) of the software application used for generation of a VR project, the interaction between the one or more media files is explained. In window202, nodes204,206, and208indicate three media files, for example 360° images. Further, connecting lines210,212, and214joining the three media files indicate the relation between the three media files. As mentioned earlier, media files could be added/deleted and the relations between the media files could be defined and changed by simple user actions, including drag and drop operations. It must be understood that three media files and linear relations between them have been illustrated and explained for the ease of illustration and explanation, and the desired number of media files could be added, and any form of relation could be defined between them, as per the requirement.

The software application running on the system102may generate a VR project. The VR project may include the one or more media files and a VRXF file including details of the interactions between the one or more media files. The VRXF file may be a text file and may be hosted over the server106.

The VRXF file may be hosted over the server106. The VRXF file may be accessed by programmers using processing devices, such as desktops, laptops, and tablets. The VRXF file may be inspected in detail to make any changes by accessing a Command User Interface (CUI).

A snippet of the VRXF file defining the position, rotation, and scale of a scene of a media file is provided below.

In the above-described format, the VRXF file may include all details of a VR project.

A group of user devices116may connect to the server106, through the communication network108, for accessing an optimal version of media files of the VR project. The group of user devices116may include applications and devices capable of playing VR projects, such as, desktops116-1, smartphones or tablets116-2, VR headsets116-3, and browsers116-4. The VR project may be optimal for some user devices but not all because user devices manufactured by same or different vendors may have different configurations. Configuration of each user device of the group of user devices116may include hardware capabilities, firmware/software type and version, and Degrees of Freedom (DoF) available for interaction. The hardware capabilities may correspond to one or more of processing power, display type and resolution, data compression-decompression capability, storage volume, and communication capability. An increase in DoF available for interaction increases quality of immersive experience obtained by a user. For example, a user device supporting 6-DoF would provide a better VR experience than a user device supporting 3-DoF.

The configurations may need to be considered for the generation of different versions of media files of the VR project, optimal for each user device. The configurations of the group of user devices116may be available with the server106. The server106may create several versions of media files of the VR project by processing the media files based on the different configurations/specifications of the group of user devices116. Processing of the media files based on the configuration of each user device may include modifying resolutions of the media files, modifying orientations of the media files, and compression of the media files.

Different versions of media files of the VR project described to be generated above may be stored alongside a VRXF file in a VR project package. The VRXF file may be a text file including details of the interaction/relation between the media files. The VR project package may be hosted over the server106.

The package of a VR project may be accessed using the group of user devices116. When a user device connects with the server106to access an optimal version of the media file of the VR project, the server106may determine if an access request of the user device fulfills the privacy requirements of a publisher of the VR project package. When a link to a VR project is accessed, the server106checks privacy settings associated with the VR project. If the VR project should no longer be accessible, the server106prevents its display. This ensures that a publisher of the VR project maintains control over accessibility of his project and manage privacy requirements effectively.

After it is determined that the user device fulfills the privacy requirements, the server106may identify the optimal version of media file of the VR project for the user device. Post identification, the server106may transmit the optimal version of the media files of the VR project to the user device, through the communication network108.

In one implementation, the VR project package may include a media files version 1 having a 720p (HD) resolution, a media files version 2 having a Full HD (1080p) resolution, a media files version 3 having a 4K (Ultra HD) resolution, and a media files version 4 having an 8K (Full Ultra HD) resolution. Further, a user device116-1may be compatible to play a 720p media files, a user device116-2may be compatible to play a Full HD media file, a user device116-3may be compatible to play a 4K media file, and a user device116-4may be compatible to play an 8K media file. In such case, when the user device116-2tries to access an optimal version of the media file of the VR project, the user device116-2may be provided an access to the media files version 2 having the Full HID (1080p) resolution. Similarly, when the user device116-4tries to access an optimal media file, the user device116-4may be provided an access to the media files version 4 having the 8K resolution.

FIG.3illustrates a first network implementation for creation and management of VR projects, in accordance with an embodiment of present invention. The first network implementation includes a first processing device102, a second processing device106, and a user device116. In present network implementation and other network implementations described henceforth, the first processing device102refers to a device accessed by a VR project creator, the second processing device106refers to a cloud based data processing device such as the server106, and the user device116refers to a device using which a user i.e. a viewer gains a VR experience. The first processing device102may execute a VR project creation component302. The VR project creation component302may be a software application installed on the first processing device102for creation of VR projects. In an alternate implementation, the VR project creation component302may be a software application hosted over a network cloud, such as the second processing device106. The VR project creator may access the VR project creation component302to select media files stored in the memory112of the first processing device102. The VR project creator may further define interactions between the media files for generation of a VR project. By processing the media files based on the interactions defined between them, the first processing device102may create the VR project. The VR project denotes the media files and a VRXF file including references i.e. network locations of the media files along with the interactions specified between the media files. By including the references of the media files instead of the content of the media files, size of the VRXF file would reduce significantly and its transmission would occur quickly.

The VRXF file may be transferred to the second processing device106. The second processing device106may include a project management component306, a content storage and processing component308, and a content distribution component310. In some implementations, the media files may be shared along with the VRXF file to the content storage and processing component308. The project management component306may manage receipt of the VRXF file by the content storage and processing component308and distribution of the VRXF file by the content distribution component310. The content storage and processing component308may process the VRXF file or the media file associated with the VRXF file to perform one or more operations, such as upscaling or downscaling of the media files. The content storage and processing component308may process the media files to generate multiple versions of the media files which may be suitable for different configurations of different user devices. For example, the content storage and processing component308may produce different versions of the media files having different resolutions. The different versions of the media files may be generated before or after a user request to access the VRXF file is received by the second processing device106.

When the user request to access the VRXF file is received from the user device116, the content distribution component310may provide the VRXF file to a VR playing component312of the user device116. The VR playing component312may be a software package having configuration information required to understand and execute the VRXF file. Such configuration information may be similar to the configuration information of the VR project creation component used for creation of the VRXF file. The VR playing component312may be developed for use by all types of the user device116or may be developed specifically for use on a particular type or brand of the user device116, such as VR headset developed by HTC®. The VR playing component312may also be configured to collect analytical playback data related to the VR projects for gaining access and playback information, to further improve creation and distribution of the VR projects. Further, security policies adhering to corporate, institutional, and/or government agencies could also be predefined within the VR playing component312. The VR player component312may encrypt media files downloaded on the user device116to ensure that the media files are not plagiarized. Further, the VR player component312may remotely remove unpublished VR projects from all user devices.

In preferred implementations, the content distribution component310may provide access of an optimal version of media files to the user device116. The optimal version of media files may be determined based on the configuration details of the user device116. For example, when the second processing device106determines that the user device116is a VR headset116-3capable of displaying 4K VR content, 4K media files may be provided to the VR headset116-3along with the VRXF file. The VR playing component312may execute the VRXF file to provide a VR experience to the user. In different implementations, the configuration information required to understand and execute the VRXF file may be provided within a firmware for example, as a part of an Operating System (OS) or the browser116-4. In such implementations, the user device116running such OS and/or the browser116-4, such as the desktop116-1may be able to execute the VRXF file without requiring to separately download and install the VR playing component312.

The above described first network implementation allows usage of media files stored locally i.e. within the first processing device102itself which is used for creation of VR projects. In this manner, more control is available over the media files used in the VR projects. Further, the first network implementation provides turnkey solutions i.e. VR project creation facility can be easily integrated with existing system(s) of an organization.

FIG.4illustrates a second network implementation for creation and management of VR projects, in accordance with an embodiment of present invention. Within the second network implementation, the media files used for creation of the VRXF file are present over internet402, instead of the memory112of the first processing device102. References such as URLs of the media files may be used by the VR project creation component302for creation of the VRXF file. In this manner, any content publicly available over the internet402could be referenced for creation of the VRXF file.

Successively, the VRXF file may be transferred to the user device116through the second processing device106. The VR playing component312running on the user device116may execute the VRXF file to provide the VR experience to the user.

By utilizing the second network implementation, consumption of the memory112is reduced as the media files present over the internet402are used. Further, an author of the media files manages access of content through enforcement of desired security policies. The author corresponds to a person owning the media files, for example a copyright holder of the media files. The media files can be distributed on a network satisfying requirements of the author e.g. a networked node nearest to intended users/target consumers. Also, access or restriction to access of content based on geolocation and security and/or government policies can be implemented.

FIG.5illustrates a third network implementation for creation and management of VR projects, in accordance with an embodiment of present invention. Some of the media files used for creation of the VRXF file may be present over the internet402and some of the media files may be stored in the memory112of the first processing device102. The VR project creation component302may utilize references of the media files required for generation of the VR project. The first processing device102may also include a text editing component502. The text editing component502may allow for manual creation or editing of a VRXF file according to the VRXF specification and reference the associated media files. The first processing device102may also include a third party component504having plugins for receiving VR projects created on other processing devices/systems and/or itself creating VR projects. The first processing device102may also include generative AI models506trained to produce a VRXF file and media files or any other content required for generation of the VR projects. VRXF file generated or edited using the VR project creation component302, the text editing component502, the third party component504, and the generative AI models506may be transferred to the project management component306through Application Programming Interfaces (APIs).

By utilizing the third network implementation, the author of the media files may be able to manage access of content through enforcement of desired security policies. The media files can be distributed on a network satisfying requirements of the author e.g. a networked node nearest to intended users/target consumers. Access or restriction to access of content based on geolocation and security and/or government policies can be implemented. The VR projects could be created by any person on his system and shared with the third party component504. Further, data available for creation of the VR projects does not remain limited to the media files, instead, required media files could be created using the generative AI models.

In above described network implementations, the first processing device102and the second processing device106have been shown separately and their functionalities have been described separately to clearly explain, in a sequential manner, different steps that occur from creation of VR projects till distribution of VR projects. However, it must be understood that the functionalities of the first processing device102and the second processing device106could be configured on a single processing device, such as the second processing device106itself. Further, the project management component306, the content storage and processing component308, and the content distribution component310have been shown to be configured within a single processing device i.e. the second processing device106. In different implementations, such components may be configured on different systems and may be managed by a single party or different parties. In such implementations, storage, management, and distribution of VR projects could be decentralized and suited as per requirements of an author of the media files, creator of the VR projects, and the user of the VR projects. Further, a Data Rights Management (DRM) service could be integrated within any of the project management component306, the content storage and processing component308, or the content distribution component310. Integration of the DRM service would allow an author to control access of the media files and a creator to control access the VR projects.

In some network implementations, the first processing device102may allow optimization of the media files, modification of hue/saturation of the media files, scaling of image content and quality, application of filters, greenscreen keying for creation of VR projects. Further, the first processing device102may offer Artificial Intelligence (AI) assisted optimization of complex source 3D models to be consumable in an immersive environment. For example, complex Computer Aided Design (CAD) models include several data points which are not relevant for immersive viewing and appropriate for fast data transfer across networked devices. Such CAD models could be optimized for creation of VR projects. The first processing device102may also allow trimming and audio level mixing of audio and video content to only include parts needed for the VR projects. Such processing of the audio and video content would reduce data requirements of the VR projects and eliminate the need learn or use complex video and audio editing software applications. The first processing device102may further allow patching of 360° image and video content to extract the required visual parts. Such processing allows better layering of content in complex 360° environments and reduces the overall size of the 360° image and video content for transfer and playback of 360° content. The first processing device102may offer audio-assisted authoring for execution of actions based on audio commands. Audio-assisted authoring can be very quick compared to usage of the input means110. Audio-assisted authoring makes discovery of authoring features more natural to inquisitive nature of humans.

The generative AI models506may be used for generation of project templates with provided prompts and media files. AI may also be used for generation of navigation prompts throughout a VR project based on content of 360° video or 360° image. For example, a door detected in one image may lead to another 360° image having a similar door, but from other side. AI would generate an interaction between the 2 data points i.e. the doors to allow a user to navigate between them.

Post creation of a VR project, the VR project creator104may allow preview of a VR project, based on configurations of the group of user devices116. In this manner, the VR project creator104may be able to view how someone using a specific user device may experience the VR project.

Referring toFIG.6, a flow chart of a method of creating a VR project is explained, in accordance with one exemplary embodiment of the present disclosure. The order in which the flow diagram for creating the VR project is described should not be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the flow diagram or alternate methods. Additionally, individual blocks may be deleted from the flow diagram without departing from the spirit and scope of the subject matter described herein. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof.

At step602, media files may be received. The media files may be received by a system operated by a user. Specifically, the media files may be received on a software application running on the system. The media files may be provided from a local storage or a remote storage. Further, the media files may be any of 360° and 2D video file, 360° and 2D image file, audio file, and 3D model file.

At step604, details of interaction between the media files may be received from the user. The details of interaction may include, for example, sequence, position, orientation, distance, and behavior of the media files.

At step606, a VR project may be created. The VR project may include different versions of the media files. The different versions of the media files may be generated through processing of the media files based on details of the interaction between the media files and predefined configurations of different user devices. Processing of the media files based on the configurations of the user devices may include modifying the resolution of the media files, modifying the orientation of the media files, and compression of the media files.

Referring toFIG.7, a flow chart of a method of identifying and executing optimal versions of media files of a VR project is explained, in accordance with one exemplary embodiment of the present disclosure. The order in which the flow diagram for identifying and executing optimal versions of media files of a VR project is described should not be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the flow diagram or alternate methods. Additionally, individual blocks may be deleted from the flow diagram without departing from the spirit and scope of the subject matter described herein. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof.

At step702, a user device may place a request to access a VR project. The request may be shared with a server hosting multiple versions of media files of the VR project. The multiple versions of the media files of the VR projects may be produced by modifying resolution and/or orientation, and compression of media files associated with the VR project, based on predefined configurations of different user devices that may access the VR project.

At step704, the server may identify an optimal version of the media files associated with the VR project, for the user device. The server may identify the optimal version of the media files based on the configuration of the user device, such as hardware capabilities (processing, display, storage, and communication capability) and firmware type and version of the user device.

At step706, the server may transmit the optimal version of the media files and a corresponding VRXF file of the VR project to the user device. The VRXF file may include details of interactions between the media files.

At step708, the user device may play the optimal version of the media files based on the details present in the VRXF file, to deliver a best VR experience of the VR project to the user.

As used in this application, the terms “component,” “system,” “platform,” “station,” “node,” “interface” are intended to refer to a computer-related entity or an entity related to, or that is part of, an operational apparatus with one or more specific functionalities, wherein such entities can be either hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to being, a process running on a processor, a processor, a hard disk drive, multiple storage drives (of optical or magnetic storage medium) including affixed (e.g., screwed or bolted) or removable affixed solid-state storage drives; an object; an executable; a thread of execution; a computer-executable program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers. Also, components as described herein can execute from various computer readable storage media having various data structures stored thereon. The components may communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems via the signal). As another example, a component can be an apparatus with specific functionality provided by mechanical parts operated by electric or electronic circuitry which is operated by a software or a firmware application executed by a processor, wherein the processor can be internal or external to the apparatus and executes at least a part of the software or firmware application. As yet another example, a component can be an apparatus that provides specific functionality through electronic components without mechanical parts, the electronic components can include a processor therein to execute software or firmware that provides at least in part the functionality of the electronic components. As further yet another example, interface(s) can include input/output (I/O) components as well as associated processor, application, or Application Programming Interface (API) components. While the foregoing examples are directed to aspects of a component, the exemplified aspects or features also apply to a system, platform, interface, layer, controller, terminal, and the like. The user device refers to devices or applications (mobile applications or desktop applications such as browsers or dedicated applications) capable of playing VR files to provide an immersive experience to users.

The server may be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a virtual environment, a mainframe computer, a network server, or a cloud-based computing environment.

The communication network providing connection between the user device and the server may be a wireless communication network using a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with each other. Further, the communication network may include a variety of network devices including routers, switches, bridges, gateways, and the like.