Method and apparatus for improvements to moving picture experts group network based media processing

A method of processing media content in Moving Picture Experts Group (MPEG) Network Based Media Processing (NBMP) may include obtaining, from an NBMP source, a workflow having a workflow descriptor (WD) indicating a workflow descriptor document (WDD); based on the workflow, obtaining a task having a task descriptor (TD) indicating a task descriptor document (TDD); based on the task, obtaining, from a function repository, a function having a function descriptor (FD) indicating a function descriptor document (FDD); and processing the media content, using the workflow, the task, and the function.

BACKGROUND

The Moving Picture Experts Group (MPEG) Network Based Media Processing (NBMP) project has developed a concept of processing media on cloud. However, current NBMP design does not provide an application program interface (API) abstraction for network management. Current NBMP design only provides APIs for cloud resources such as a hardware platform.

SUMMARY

According to embodiments, a method of processing media content in Moving Picture Experts Group (MPEG) Network Based Media Processing (NBMP) is performed by at least one processor, and includes obtaining, from an NBMP source, a workflow for processing the media content, the workflow having a workflow descriptor (WD) indicating a workflow descriptor document (WDD); based on the workflow, obtaining a task for processing the media content, the task having a task descriptor (TD) indicating a task descriptor document (TDD); based on the task, obtaining, from a function repository storing one or more functions for processing the media content, at least one among the one or more functions, each of the at least one among the one or more functions having a function descriptor (FD) indicating a function descriptor document (FDD); and processing the media content, using the workflow, the task, and the at least one among the one or more functions.

According to embodiments, an apparatus for processing media content in Moving Picture Experts Group (MPEG) Network Based Media Processing (NBMP) includes at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, which includes first obtaining code configured to cause the at least one processor to obtain, from an NBMP source, a workflow for processing the media content, the workflow having a workflow descriptor (WD) indicating a workflow descriptor document (WDD); second obtaining code configured to cause the at least one processor to obtain, based on the workflow, a task for processing the media content, the task having a task descriptor (TD) indicating a task descriptor document (TDD); third obtaining code configured to cause the at least one processor to obtain, based on the task, from a function repository storing one or more functions for processing the media content, at least one among the one or more functions, each of the at least one among the one or more functions having a function descriptor (FD) indicating a function descriptor document (FDD); and processing code configured to cause the at least one processor to process the media content, using the workflow, the task, and the at least one among the one or more functions.

According to embodiments, a non-transitory computer-readable medium stores instructions that, when executed by at least one processor of an apparatus for processing media content in Moving Picture Experts Group (MPEG) Network Based Media Processing (NBMP), cause the at least one processor to obtain, from an NBMP source, a workflow for processing the media content, the workflow having a workflow descriptor (WD) indicating a workflow descriptor document (WDD); based on the workflow, obtain a task for processing the media content, the task having a task descriptor (TD) indicating a task descriptor document (TDD); based on the task, obtain, from a function repository storing one or more functions for processing the media content, at least one among the one or more functions, each of the at least one among the one or more functions having a function descriptor (FD) indicating a function descriptor document (FDD); and process the media content, using the workflow, the task, and the at least one among the one or more functions.

DETAILED DESCRIPTION

Embodiments described herein provide functional improvements to the MPEG NBMP standard. Such improvements increase media processing efficiency, increase speed and lower cost of deployment of media services, and allow large scale deployment of media services by leveraging public, private or hybrid cloud services.

In examples, the functional improvements to the MPEG NBMP standard include a harmonization of Workflow, Task and Functions, and defining a one-to-one relationship between logical items, data documents and REST resources for each of them. Access to main and basic descriptors may be enabled by creating representational state transfer (REST) resources for every descriptor. Also, a Task Descriptor may be added to a Workflow Descriptor, so that the Workflow descriptor can capture the complete and full picture of a workflow and no additional information is needed to build or examine the workflow.

In addition, a Task life-cycle state may be added as a parameter to a Task's General Descriptor, so that Task Descriptor may also capture the state of each Task and the state information may be retrieved using existing NBMP Workflow and TASK APIs. Examples also may including improving NBMP API design by making them REST APIs. The request and responses may be re-defined to comply with a REST concept, and therefore achieve the benefits of REST.

In addition, embodiments may simplify the function discovery operation by making it a query-string search, and allow multiple key-value pairs to be added in query string, enabling more extensive and powerful search compared to the current NBMP design.

FIG.1is a diagram of an environment100in which methods, apparatuses and systems described herein may be implemented, according to embodiments. As shown inFIG.1, the environment100may include a user device110, a platform120, and a network130. Devices of the environment100may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections.

The user device110includes one or more devices capable of receiving, generating, storing, processing, and/or providing information associated with platform120. For example, the user device110may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smart phone, a radiotelephone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), or a similar device. In some implementations, the user device110may receive information from and/or transmit information to the platform120.

The platform120includes one or more devices as described elsewhere herein. In some implementations, the platform120may include a cloud server or a group of cloud servers. In some implementations, the platform120may be designed to be modular such that software components may be swapped in or out depending on a particular need. As such, the platform120may be easily and/or quickly reconfigured for different uses.

In some implementations, as shown, the platform120may be hosted in a cloud computing environment122. Notably, while implementations described herein describe the platform120as being hosted in the cloud computing environment122, in some implementations, the platform120may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

The cloud computing environment122includes an environment that hosts the platform120. The cloud computing environment122may provide computation, software, data access, storage, etc. services that do not require end-user (e.g., the user device110) knowledge of a physical location and configuration of system(s) and/or device(s) that hosts the platform120. As shown, the cloud computing environment122may include a group of computing resources124(referred to collectively as “computing resources124” and individually as “computing resource124”).

The computing resource124includes one or more personal computers, workstation computers, server devices, or other types of computation and/or communication devices. In some implementations, the computing resource124may host the platform120. The cloud resources may include compute instances executing in the computing resource124, storage devices provided in the computing resource124, data transfer devices provided by the computing resource124, etc. In some implementations, the computing resource124may communicate with other computing resources124via wired connections, wireless connections, or a combination of wired and wireless connections.

As further shown inFIG.1, the computing resource124includes a group of cloud resources, such as one or more applications (“APPs”)124-1, one or more virtual machines (“VMs”)124-2, virtualized storage (“VSs”)124-3, one or more hypervisors (“HYPs”)124-4, or the like.

The application124-1includes one or more software applications that may be provided to or accessed by the user device110and/or the platform120. The application124-1may eliminate a need to install and execute the software applications on the user device110. For example, the application124-1may include software associated with the platform120and/or any other software capable of being provided via the cloud computing environment122. In some implementations, one application124-1may send/receive information to/from one or more other applications124-1, via the virtual machine124-2.

The virtual machine124-2includes a software implementation of a machine (e.g., a computer) that executes programs like a physical machine. The virtual machine124-2may be either a system virtual machine or a process virtual machine, depending upon use and degree of correspondence to any real machine by the virtual machine124-2. A system virtual machine may provide a complete system platform that supports execution of a complete operating system (“OS”). A process virtual machine may execute a single program, and may support a single process. In some implementations, the virtual machine124-2may execute on behalf of a user (e.g., the user device110), and may manage infrastructure of the cloud computing environment122, such as data management, synchronization, or long-duration data transfers.

The hypervisor124-4may provide hardware virtualization techniques that allow multiple operating systems (e.g., “guest operating systems”) to execute concurrently on a host computer, such as the computing resource124. The hypervisor124-4may present a virtual operating platform to the guest operating systems, and may manage the execution of the guest operating systems. Multiple instances of a variety of operating systems may share virtualized hardware resources.

FIG.2is a block diagram of example components of one or more devices ofFIG.1. The device200may correspond to the user device110and/or the platform120. As shown inFIG.2, device200may include a bus210, a processor220, a memory230, a storage component240, an input component250, an output component260, and a communication interface270.

The bus210includes a component that permits communication among the components of the device200. The processor220is implemented in hardware, firmware, or a combination of hardware and software. The processor220is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or another type of processing component. In some implementations, the processor220includes one or more processors capable of being programmed to perform a function. The memory230includes a random access memory (RAM), a read only memory (ROM), and/or another type of dynamic or static storage device (e.g., a flash memory, a magnetic memory, and/or an optical memory) that stores information and/or instructions for use by the processor220.

The input component250includes a component that permits the device200to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, and/or a microphone). Additionally, or alternatively, the input component250may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and/or an actuator). The output component260includes a component that provides output information from the device200(e.g., a display, a speaker, and/or one or more light-emitting diodes (LEDs)).

The communication interface270includes a transceiver-like component (e.g., a transceiver and/or a separate receiver and transmitter) that enables the device200to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface270may permit the device200to receive information from another device and/or provide information to another device. For example, the communication interface270may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, or the like.

The device200may perform one or more processes described herein. The device200may perform these processes in response to the processor220executing software instructions stored by a non-transitory computer-readable medium, such as the memory230and/or the storage component240. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.

Software instructions may be read into the memory230and/or the storage component240from another computer-readable medium or from another device via the communication interface270. When executed, software instructions stored in the memory230and/or the storage component240may cause the processor220to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

The number and arrangement of components shown inFIG.2are provided as an example. In practice, the device200may include additional components, fewer components, different components, or differently arranged components than those shown inFIG.2. Additionally, or alternatively, a set of components (e.g., one or more components) of the device200may perform one or more functions described as being performed by another set of components of the device200.

FIG.3is a block diagram of an NBMP system300, according to embodiments.

Referring toFIG.3, the NBMP system300includes an NBMP source310, an NBMP workflow manager320, a function repository330, a network controller340, one or more media processing entities350, a media source360, and a media sink370.

The NBMP source310may receive instructions from a third party entity380, may communicate with the NBMP workflow manager320via an NBMP workflow API, and may communicate with the function repository330via a function discovery API. For example, the NBMP source310may send a workflow description document to the NBMP workflow manager320, and may read a function description of functions that are stored in a memory of the function repository330. The functions may include media processing functions such as, for example, functions of media decoding, feature point extraction, camera parameter extraction, projection method, seam information extraction, blending, post-processing, and encoding. The NBMP source310may include at least one processor and a memory that stores code configured to cause the at least processor to perform functions of the NBMP source310.

The NBMP source310may request the NBMP workflow manager320to create workflow including tasks351and352to be performed by the one or more media processing entities350, by sending the workflow description document to the NBMP workflow manager320. The workflow description document may include descriptors, each of which may include parameters.

For example, the NBMP source310may select one or more of the functions stored in the function repository330, and send, to the NBMP workflow manager320, the workflow description document including the descriptors for describing details such as input and output data, the selected one or more of the functions, and requirements for a workflow. The workflow description document may further include a set of task descriptions and a connection map of inputs and outputs of the tasks351and352to be performed by the one or more of the media processing entities350. When the NBMP workflow manager320receives such information from the NBMP source310, the NBMP workflow manager320may create the workflow by instantiating the tasks351and352based on function names and connecting the tasks351and352in accordance with the connection map.

Alternatively or additionally, the NBMP source310may request the NBMP workflow manager320to create a workflow by using a set of keywords. For example, the NBMP source310may send, to the NBMP workflow manager320, the workflow description document including the set of the keywords that the NBMP workflow manager320may use to find appropriate one or more of the functions stored in the function repository330. When the NBMP workflow manager320receives such information from the NBMP source310, the NBMP workflow manager320may create the workflow by searching for the appropriate one or more of the functions, using the keywords that may be specified in a Processing Descriptor of the workflow description document, and by using other descriptors in the workflow description document to provision and connect the tasks351and352.

The NBMP workflow manager320may communicate with the function repository330via a function discovery API, and may communicate with one or more of the media processing entities350, through the network controller340, via an NBMP task API, an NBMP link API, and a function discovery API. The NBMP workflow manager320may include at least one processor and a memory that stores code configured to cause the at least processor to perform functions of the NBMP workflow manager320.

The NBMP workflow manager320may use the NBMP task API to setup, configure, manage, and monitor one or more of the tasks351and352of the workflow that is performable by the one or more media processing entities350. In embodiments, the NBMP workflow manager320may use the NBMP task API to update and destroy the tasks351and352. To configure, manage, and monitor the tasks351and352of the workflow, the NBMP workflow manager320may send messages, such as requests, to one or more of the media processing entities350, wherein each message may have descriptors, each of which may include parameters. The tasks351and352may each include one or more media processing functions354and one or more configurations353for the one or more media processing functions354.

In embodiments, after receiving the workflow description document from the NBMP source310that does not include a list of tasks (e.g., includes a list of keywords instead of a list of tasks), the NBMP workflow manager320may select the tasks based on descriptions of the tasks in the workflow description document, to search the function repository330, via the function discovery API, to find appropriate one or more of the functions to run as the tasks351and352for the current workflow. For example, the NBMP workflow manager320may select the tasks based on keywords that are provided in the workflow description document. After the appropriate one or more of the functions are identified using the keywords or the set of task descriptions that is provided by the NBMP source310, the NBMP workflow manager320may configure the selected tasks in the workflow by using the NBMP task API. For example, the NBMP workflow manager320may extract configuration data from information that is received from the NBMP source, and configure the tasks351and352based on the extracted configuration data.

The one or more media processing entities350may be configured to receive media content from the media source360, process the received media content in accordance with the workflow that includes the tasks351and352and is created by the NBMP workflow manager320, and output the processed media content to the media sink370. The one or more media processing entities350may each include at least one processor and a memory that stores code configured to cause the at least processor to perform functions of the one or more media processing entities350.

The network controller340may include at least one processor and a memory that stores code configured to cause the at least processor to perform functions of the network controller340.

The media source360may include memory that stores media and may be integrated with or separate from the NBMP source310. In embodiments, the NBMP workflow manager320may notify the NBMP source310and/or the media source360when a workflow is prepared, and the media source360may transmit media content to the one or more of the media processing entities350based on a notification that the workflow is prepared.

The media sink370may include at least one processor and at least one display that is configured to display media content that is processed by the one or more media processing entities350.

The third party entity380may include at least one processor and a memory that stores code configured to cause the at least processor to perform functions of the third party entity380.

As discussed above, messages from the NBMP source310(e.g. a workflow description document for requesting creation of a workflow) to the NBMP workflow manager320, and messages (e.g. for causing the workflow to be performed) from the NBMP workflow manager320to the one or more media processing entities350may include descriptors, each of which including parameters. In embodiments, communication between any of components of the NBMP system300using an API may include descriptors, each of which including parameters.

Harmonization of Workflow, Task and Function Hierarchal Objects and Aligning Them as REST Resources

The current NBMP specification has a concept of template for task and functions, while defining a document for workflow description referred to as a workflow description document (WDD). Furthermore, it is not clear how task and function templates are converted to REST resources. The API documentation does not clearly define the exact syntax of resources, or how what resources are included in acknowledge of a API operation.

An NBMP specification according to embodiments aligns the concepts of logical items, JOSN objects/XML documents, and REST resources for the three items: workflow, task and function, and builds a harmonized and aligned structure for all there. Furthermore, it defines the resource format and constraints in acknowledgements, to make the interfaces true REST APIs.

FIG.4shows an example of the relationship between the three entities, as logical items, JSON/XML documents and REST resources.

As can be seen inFIG.4, logical items may be defined as descriptors. The three main descriptors may be a workflow desciptor (WD), a task descriptor (TD) and a function descriptor (FD). WD, TD, and FD may be built as combinations of the basic descriptors, such as a general descriptor, an input descriptor, an output descriptor, and others.

Workflow description objects (WO), task description objects (TO), and function description objects (FO) may be code realizations of the corresponding logic items, for example, WD, TD, and FD, in JSON or XML.

WDD, task description documents (TDD), and function description documents (FDD) may be documents containing WOs, TOs, and FOs. Documents may be objects in JSON and XML document in XML. Note that FDD may differ from WDD and TDD in the sense that it may include one or more FOs.

Workflow resources (WR), task resources (TR), and function resources (FR) may be WDD, TDD, or FDD includes URLs, and therefore may be identified as REST resources.

A major benefit of the example NBMP specification discussed above is that there may be a one to one relationship between main descriptors, documents, and REST resources, which may allow the system to be precisely specified, and may allow interoperable solutions to be built accordingly.

Designating the Main and Basic Descriptors as REST Resources

The current NBMP specification only enables accessing WR, TR and FR.

An NBMP specification according to embodiments may make WR, TR, and FR, REST resources, and also enable the main and basic descriptors to become a REST resource. Therefore, these descriptors may be individually accessed using the NBMP APIs. In this design, each main and basic descriptor object of an included WDD, TDD and FDD in any response may include include one “link” object, including a “ref” with value “self” and a URL indicating the location to that object.

Adding the Task Descriptor as a Component of Workflow Descriptor

The current NBMP specification does not include the Task Descriptor as part of workflow. It defines the relationship between them using ConnectionMap descriptor, which also includes the function indentifiers.

However, for an NBMP source to have a complete workflow diagram, an NBMP specification according to embodiments may include task descriptors also in the workflow descriptor, as shown for example in Table 1.

In Table 1, an additional item, task descriptor, is added. With this addition, a workflow descriptor may describe the complete map and information about a created workflow.

Adding Media Source and Sink to Workflow DAG

The current NBMP specification does not include the media source360and sink370in its workflow directed acyclic graph (DAG) description.

An NBMP specification according to embodiments adds these elements to workflow DAG. A benefit of this approach is that the resource requirements for network connection between media source360and workflow in on hand, and between Workflow and media sink370can be described with the same DAG. This approach simplifies the documentation of requirements as well as the establishment and management of workflow by NBMP Workflow Manager.

Adding the Task's Life-Cycle State to General Descriptor

The current NBMP specification defines a life-cycle for tasks. The life-cycle has 5 states. However, the state of the current state of a Task in not described in any descriptors.

An NBMP specification according to embodiments builds based on REST APIs. Each REST resource must also capture its state. Therefore, the resource maintains the complete status of the logical item and there is no need to acquire the state from other data structure.

The NBMP specification according to embodiments adds a “state” parameter to general descriptor, which can be used to describe a task's state. This addition is shown in Table 2.

As shown in Table 2, and as illustrated inFIG.3, a new state “Null” may also added to the life-cycle to capture the initial state of a Task in its life cycle.

Designate NBMP APIs as REST APIs

The current NBMP specification is not clear how the API operations, requests and responses work. The documentation does not define interoperable APIs at this stage.

An NBMP specification according to embodiments designs NBMP APIs as REST API. Therefore, all API operations may be implemented as REST methods using HTTP 1.1, and the requests and responses may include REST resources, as shown inFIG.2. Furthermore, the HTTP status code is used.

Tables 3, 4 and 5 describes the improved NBMP APIs, their requests and responses.

TABLE 3Improved Workflow API OperationsRequest resourceOperationDescriptionrequirementsResponse RequirementsCreateWorkflowCreate aWR includingIf successful, shall include:workflowinformation needed to1) HTTP status code 2xxcreate a workflow2) Response's body with updated WRThe Generalincluding:descriptor's Id shall nota) A value for General descriptor'sbe included in thisIdrequestb) Updated information includingendpoint information where tosend media data, metadata, andother informationIf failed, shall include:1) HTTP status codes 4xx or 5xx2) Optionally, response's body withupdated WR signaling faileddescriptors or parametersUpdateWorkflowUpdate aUpdated WR withIf successful, shall include:previouslyidentical General's Id,1) HTTP status code 2xxcreatedpreviously received in2) Response's body with updated WRworkflowCreateWorkflow'sincluding:responseGeneral descriptor's Id identicalto the one in the requestUpdated information includingendpoint information where tosend media data, metadata, andother informationIf failed, shall include:1) HTTP status codes 4xx or 5xx2) Response's body with updated WRsignaling failed descriptors orparametersDeleteWorkflowTerminate aWR with identicalIf successful, shall include: HTTPpreviouslyGeneral's Id,status code 2xxcreatedpreviously received inIf failed, shall include:workflowCreateWorkflow's1) HTTP status codes 4xx or 5xxresponse2) Response's body with updated WRsignaling failed descriptors orparametersRetrieveWorkflowRetrieve aWR with identicalIf successful, shall include:previouslyGeneral's Id,1) HTTP status code 2xxconfiguredpreviously received in2) Response's body with updated WRworkflowCresponsereateWorkflow'sincluding:responseGeneral descriptor's Id identicalto the one in the requestUpdated information includingendpoint information where tosend media data, metadata, andother informationIf failed, shall include:1) HTTP status codes 4xx or 5xx2) Response's body with updated WRsignaling failed descriptors orparametersGetReportsGet reportsWR with identicalIf successful, shall include:for aGeneral's Id,1) HTTP status code 2xxpreviouslypreviously received in2) Response's body with updated WRcontiguredCreateWorkflow'sincluding:workflowresponse and ReportGeneral descriptor s Id identicaldescriptorsto the one in the requestUpdated report descriptors whichwere included in the requestIf failed, shall include:1) HTTP status codes 4xx or 5xx2) Response's body with updated WRsignaling failed descriptors orparameters

TABLE 4Improved Task Configuration OperationsRequestOperationDescriptionParametersResponse RequirementsCreateTaskProvision toTR includingIf successful, shall include:run a taskinformation3) HTTP status code 2xxinside theneeded to create a4) Response's body with updated TR including:mediaworkflowa) A value for General descriptor's IdprocessingThe Generalb) Updated information including endpointentitydescriptor's Idinformation where to send media data,shall not bemetadata, and other informationincluded in thisIf failed, shall include:request.3) HTTP status codes 4xx or 5xxOptionally, response's body with updated TRsignaling failed descriptors or parametersUpdateTaskModify theUpdated TR withIf successful, shall include:task runningidentical3) HTTP status code 2xxinside theGeneral's Id,4) Response's body with updated TR including:mediapreviouslyGeneral descriptor's Id identical to the oneprocessingreceived inin the requestentityCreateTask'sUpdated information including endpointresponseinformation where to send media data,metadata, and other informationIf failed, shall include:3) HTTP status codes 4xx or 5xx4) Response's body with updated TR signallingfailed descriptors or parametersGetTaskRetrieve taskTR with identicalIf successful, shall include:configurationGeneral's Id,3) HTTP status code 2xxinformationpreviously4) Response's body with updated TR including:received inGeneral descriptor's Id identical to the oneCreateTask'sin the requestresponseUpdated report descriptors which wereincluded in the requestIf failed, shall include:3) HTTP status codes 4xx or 5xxResponse's body with updated TR signallingfailed descriptors or parametersDeleteTaskRequest toTR with identicalIf successful, shall include:de-provisionGeneral's Id,1. HTTP status code 2xxthe taskreceived inIf failed, shall include:inside theCreateTask'sHTTP status codes 4xx or 5xxmediaresponse.Response's body with updated TRprocessingsignalling failed descriptors or parametersentity

In Tables 3-5, the request and response body (data) may correspond to the REST resources ofFIG.2.

Additionally, to make the response a complete REST resource, an NBMP specification according to embodiments may specify that the included WDD in any response may include one “link” object, including a “ref” with value “self” and a URL indicating the location to WDD. In addition, the included TDD in any response may include one “link” object, including a “ref” with value “self” and a URL indicating the location to TDD. Similarly, each included FDD in any response may include one “link” object, including a “ref” with value “self” and a URL indicating the location to FDD. Further, in every included FDD in any response, each Function Descriptor Object (FDO) may include one “link” object, including a “ref” with value “self” and a URL indicating the location of each FDO.

HTTP “Query String”-Based Search for Function Discovery

The current NBMP specification describes discovery operations. But it does not define the format and protocol of the operations. It also specifies that two keys are used for Function search: either by “identifier” or by “name”, but not the combination of them.

An NBMP specification according to embodiments improves the discovery operations by using HTTP query string and combination of key-value pairs. So it defines a simple method (HTTP GET) to implement the operations. This allows the combination of keys to be used in the search. In addition, an NBMP specification according to embodiments adds more keys to the search parameters. Therefore, better search can be performed using different aspects of stored Function in the repository.

According to embodiments, a discovery query and a query string may be used to perform a search. The discovery query is to discover one or more Functions in Function Repository by the properties described in the query. The query string may be used to describe these properties; and the query string may include of a set of key-value pairs, separated by a single ‘&’ character. In each key-value pair, the key and value shall be separated by single ‘=’ character. A query string may be added to the end of resource URL after a single “?” character.

Table 7 lists examples of supported keys in the query string.

FIG.6is a flowchart of a method600of processing media content in MPEG NBMP, according to embodiments. In some implementations, one or more process blocks ofFIG.6may be performed by the platform120implementing the NBMP system300. In some implementations, one or more process blocks ofFIG.6may be performed by another device or a group of devices separate from or including the platform120implementing the NBMP system300, such as the user device110.

As shown inFIG.6, in operation610, the method600includes obtaining, from an NBMP source, for example NBMP source310, a workflow for processing the media content, the workflow having a WD indicating a workflow descriptor document WDD.

In operation620, the method600includes, based on the workflow, obtaining a task for processing the media content, the task having a TD indicating a TDD.

In operation630, the method600includes, based on the task, obtaining at least one among the one or more functions from a function repository storing one or more functions for processing the media content, for example function repository330, wherein each of the at least one among the one or more functions has an FD indicating an FDD.

In operation640, the method600includes processing the media content, using the workflow, the task, and the at least one among the one or more functions.

In an embodiment, the workflow may include a workflow representational state transfer (REST) resource (WR), the task may include a task REST resource (TR), and the at least one among the one or more functions may include a function REST resource.

In an embodiment, the WD, the TD, and the FD may be constructed from one or more general descriptors.

In an embodiment, the WDD may include a workflow description object (WO), the TDD may include a task description object (TO), and the FDD may include at least one function description object (FO).

In an embodiment, the WO, the TO, and the at least one FO include at least one JavaScript Object Notation (JSON) object or at least one Extensible Markup Language (XML) element.

In an embodiment, the WDD may include first link object including a first uniform resource locator (URL) indicating a location of the WDD, the TDD may include a second link object including a second uniform resource locator (URL) indicating a location of the TDD, and the FDD may include a third link object including a third uniform resource locator (URL) indicating a location of the FDD.

In an embodiment, the TD may include a state descriptor indicating a state of the task.

In an embodiment, the state descriptor may indicate that the state of the task is a null state.

In an embodiment, the at least one among the one or more functions may be retrieved from the function repository using a hyper text transfer protocol (HTTP) query including a search key and a search value corresponding to the at least one among the one or more functions.

In an embodiment, the search value may include at least one from an identifier, a name, a description, a brand, or a keyword associated with the at least one among the one or more functions.

AlthoughFIG.6shows example blocks of the method600, in some implementations, the method600may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted inFIG.6. Additionally, or alternatively, two or more of the blocks of the method600may be performed in parallel.

FIG.7is a diagram of an apparatus700for processing media content in MPEG NBMP, according to embodiments. As shown inFIG.7, the apparatus700includes first obtaining code710, second obtaining code720, third obtaining code730, and processing code740.

The first obtaining code710may be configured to cause the at least one processor to obtain, from an NBMP source such as NBMP source310, a workflow for processing the media content, the workflow having a workflow descriptor (WD) indicating a workflow descriptor document (WDD).

The second obtaining code720may be configured to cause the at least one processor to obtain, based on the workflow, a task for processing the media content, the task having a task descriptor (TD) indicating a task descriptor document (TDD).

The third obtaining code730may be configured to cause the at least one processor to obtain, based on the task, at least one among the one or more functions from a function repository storing one or more functions for processing the media content, for example function repository330, wherein each of the at least one among the one or more functions has an FD indicating an FDD.

The processing code740may be configured to cause the at least one processor to process the media content, using the workflow, the task, and the at least one among the one or more functions.

In an embodiment, the workflow may include a workflow representational state transfer (REST) resource (WR), the task may include a task REST resource (TR), and the at least one among the one or more functions may include a function REST resource.

In an embodiment, the WD, the TD, and the FD may be constructed from one or more general descriptors.

In an embodiment, the WDD may include a workflow description object (WO), the TDD may include a task description object (TO), and the FDD may include at least one function description object (FO).

In an embodiment, the WO, the TO, and the at least one FO include at least one JavaScript Object Notation (JSON) object or at least one Extensible Markup Language (XML) element.

In an embodiment, the WDD may include first link object including a first uniform resource locator (URL) indicating a location of the WDD, the TDD may include a second link object including a second uniform resource locator (URL) indicating a location of the TDD, and the FDD may include a third link object including a third uniform resource locator (URL) indicating a location of the FDD.

In an embodiment, the TD may include a state descriptor indicating a state of the task.

In an embodiment, the state descriptor may indicate that the state of the task is a null state.

In an embodiment, the at least one among the one or more functions may be retrieved from the function repository using a hyper text transfer protocol (HTTP) query including a search key and a search value corresponding to the at least one among the one or more functions.

In an embodiment, the search value may include at least one from an identifier, a name, a description, a brand, or a keyword associated with the at least one among the one or more functions.