Patent Description:
3GPP SA4 has defined the Framework for Live uplink Streaming (FLUS) in 3GPP TS <NUM>. A guideline document is available in 3GPP TR <NUM>.

FLUS contains an IMS and a Non-IMS based instantiation. In case of the Non-IMS based instantiation the FLUS source can provision the FLUS sink using HTTP Rest APIs. The provisioning procedure (Using F-C) is openly defined. At minimum, the FLUS Source should provision the FLUS media instantiation (e.g. "org:3gpp:flus:<NUM>:instantiations:fmp4"). In return, the FLUS Source gets an Ingest Point description in form of a Push URL (e.g. http://sink. com/sessionxyz/).

<FIG> illustrates the current FLUS architecture, where the user and control planes of the FLUS source and sink are co-located on the same device. The F reference point connects a FLUS source and a FLUS sink. The F reference point enables the FLUS source to establish and control a single FLUS session. The F reference point also enables the FLUS sink and the FLUS source to mutually authenticate and authorize each other.

FLUS ctrl and F-C describe control plane functionality including the associated processing by FLUS sink of the uploaded media for subsequent downstream distribution, plus FLUS media instantiation selection. F-C may also support configuration of static metadata for the session. FLUS media and F-U describe user plane functionality which includes setup of one or more media sessions and subsequent media data transmission via media streams.

F-C is used to establish and control the FLUS session. F-C allows the FLUS source to select a FLUS media instantiation, provision static metadata associated with each media session present in the FLUS session, and select and configure the processing and distribution sub-functions. The FLUS media instantiation is defined as part of the FLUS session. The user plane (F-U) may also contain the media stream establishment procedures when needed. Multiple media streams may be established for one FLUS session. Further aspects are described in the "<NPL>, in <CIT>, and <CIT>. In "<NPL>, a method is disclosed in which a controller node controls a streaming session. A need exists to improve a flexibility in controlling a media stream.

For the Non-IMS based FLUS instantiation, there is no mechanism defined to provide commands from the FLUS Sink to the FLUS source. In embodiments, such a mechanism, e.g. a command or recommendation channel, is referred to as a "Remote Control" channel in the following description.

There is a one-to-one relation between the FLUS Media components on the FLUS Source and the FLUS Sink. However, there may be different network entities, which can remote control the FLUS source (control plane parts), e.g. in form of commands like start/stop or in form of recommendations like to increase the bitrate. Some commands may be targeted to the encoding process (e.g., increase bitrate or change resolution). Some commands may target the actual capture device (e.g., camera), such as pan or zoom. These commands might, however, come from different entities in the network, e.g., pan or zoom could originate from a remote production studio and bit-rate increase could originate from a control function inside the network. The need is met by the features of the independent claims.

The advantage provided by some embodiments is to allow different types of remote control commands from different end-points. Different end-points may be authorized to give remote control commands, including all possible commands or some limited subset of such commands. For example, a FLUS source may establish remote control channels to a first F-RC end-point, which is collocated with the F-U end-point (FLUS sink media). This F-RC end-point is (in this example) authorized to have full control over capture and the encoding processes, like start and stop. The FLUS Source (subdivided into a FLUS source media and FLUS source control, as shown in <FIG>) may establish remote control connections (F-RC) to a second end-point, which is only authorized to influence the encoding bitrate. The FLUS Source is ignoring any other commands from the second end-point. The FLUS Source may establish a remote control connection (F-RC) to a third end-point, which is only authorized to influence the camera, e.g. by way of commands affecting Pan, Zoom, shutter speed, and so forth. The F-C (or F-RC) end-points might be distributed at different locations, such as a remote production studio for capturing/stopping/zooming, and inside the network for adapting the bitrate.

Embodiments allow the instantiation of multiple FLUS Ctrl instances which are connected to one FLUS media sink and multiple F-RC instances to one FLUS Source. This extends the current FLUS architecture and allows for controlling the FLUS source from multiple end-points. Embodiments have no impact on the processing and distribution functions at the UE, which get data from the FLUS media sink. There may be different remote control channel realizations, such as WebSocket-based or HTTP-Pull-based channels. Further, there may be different remote control command sets (e.g., command sets related to camera operation, or encoding processes, or grouped in some other fashion).

According to some embodiments, there may be multiple Control Plane entities for a single FLUS User Plane entity (FLUS source media and FLUS sink media). Some Control Plane Entities may use just provisioning procedures (F-C), while some others may use "remote control" procedures (using F-RC).

Different Remote Control (F-RC) authorization levels may be defined. Some F-RC entities may only be allowed to change the encoding procedure (e.g. bitrate, resolution, codec configuration) for FLUS media. Other F-RC entities may be allowed to start/stop/pause the FLUS Media. Other levels of authorization may also be used.

In the simplest case, there is one FLUS F-C entity (without any FLUS F-RC entity) for one FLUS F-U entity. In other deployments, there may be additional (even more than one) F-RC entities per F-U located within the FLUS Source. The peer entities may be located in different nodes.

In embodiments, upon provisioning, the FLUS Source immediately establishes the F-RC (remote) control sessions. When more than one F-RC control session is configured, the FLUS Source is establishing the connection to each. A F-RC remote control connection may be realized by a web socket or other client initiated push schemes. The FLUS Source may receive time-stamped remote control commands. the FLUS sink may send a F-U media session start time prior to the event. Without a timestamp, the FLUS Source is immediately executing the received remote control command. Remote control commands may be start media session, stop media session, change codec configuration (resolution, bitrate), pause, pan or zoom of the camera, among other possible commands.

According to a first aspect, a method, performed by a FLUS source control entity, for controlling delivery of a media source to a media sink, is provided as mentioned in claim <NUM>. The method includes inter alia: the source control entity sending an indication of availability for streaming; the source control entity establishing connections with one or more controller nodes; the source control entity receiving a first command from one of the one or more controller nodes; and the source control entity relaying the first command to the media source.

In some embodiments, the method further includes the source control entity verifying that the one of the one or more controller nodes is authorized to issue the first command prior to relying the first command to the media source. In some embodiments, the one or more controller nodes includes a first controller node authorized to send commands to control a camera associated with the media source and a second controller node authorized to send commands to affect a bitrate associated with the media source. In some embodiments, the one or more controller nodes are located remotely from the source control entity.

According to a second aspect, a FLUS source control entity is provided as mentioned in claim <NUM>, being adapted to inter alia: send an indication of availability for streaming; establish connections with one or more controller nodes; receive a first command from one of the one or more controller nodes; and relay the first command to the media source.

According to a third aspect, a computer program is provided, comprising instructions which, when executed on at least one processor, causes the at least one processor to carry out the method according to any one of embodiments of the first aspect.

According to a fourth aspect, a carrier is provided comprising the computer program of the third aspect, wherein the carrier is one of an electronic signal, optical signal, radio signal or computer readable storage medium.

Further embodiments are set out in the dependent claims.

The above and other embodiments are described below.

An embodiment is now described. According to this embodiment, the FLUS framework is orchestrated via various nodes. For this example, we assume here that three nodes are controlling multiple FLUS sources, in this case the FLUS source resides in a flying drone covering an outdoor event. The involved nodes are depicted in <FIG>, and described below. More of fewer nodes are possible, and different scenarios than a flying drone covering an outdoor event are encompassed by the disclosed embodiments.

Session orchestrator <NUM>: This node <NUM> could be in the edge or the origin of the cloud, and it is responsible for announcing the addresses and privileges of different control nodes. The node <NUM> distributes the information to the FLUS source control node <NUM> via any text descriptor format (e.g., JSON/XML). The node <NUM> is considered as the entry point that connect the FLUS source <NUM> with other FLUS control units.

FC1- session management <NUM>: This node <NUM> can be in the operator or the content provider cloud. The node <NUM> is responsible for handling any changes in the FLUS control nodes (e.g. handover), and may be connected to the FLUS Source Control <NUM> via using a persistent push mechanism for remote control (e. g WebSocket). That is, the node <NUM> may be connected by a remote control connection, i.e. F-RC.

FC2- FLUS session editing controller <NUM>: This node <NUM> is responsible for managing the FLUS sources <NUM> in terms of video editing (i.e. uses a remote control connection). It can send zooming commands (such as zoom-in or zoom-out), or change a location or angle of view to a specific FLUS source <NUM>. The node <NUM> may also be responsible to pause/resume a stream according to the stream needs. The node <NUM> can be part of the content owner cloud, can be deployed in the operator edge (if low latency is needed), or can be in a third-party cloud. The node <NUM> may be connected to the FLUS Source Control <NUM> via persistent push mechanism for remote control (e. g WebSocket). That is, the node may be connected by a remote control connection, i.e. F-RC.

FC-<NUM> Drone bitrate controller <NUM>: This node <NUM> is responsible for controlling the stream bitrate. The node <NUM> may provide a suggested bitrate e.g. based on the current (or expected) load on the network. This node <NUM> can be deployed in the operator cloud and may retrieve information on network status from the operator. The node <NUM> is connected to the FLUS Source Control <NUM> via persistent push mechanism for remote control (e. g WebSocket). That is, the node <NUM> may be connected by a remote control connection, i.e. F-RC.

FLUS source controller <NUM> (called in TS <NUM> FLUS CTRL): This node <NUM> is responsible for communicating with the above-mentioned nodes, and may be deployed inside the FLUS source <NUM>. The node <NUM> may also be split into the F-C handling part (e.g. HTTP GET) and the F-RC handling parts (e.g. web socket).

FLUS source media <NUM> (called in TS <NUM> FLUS Media): This node <NUM> may send media streams to the FLUS sink <NUM>. The node <NUM> may be deployed inside the FLUS source <NUM> (e.g., the drone).

Flus sink media <NUM> (FLUS media in TS <NUM>): This node <NUM> may receive media streams from the FLUS source <NUM>.

<FIG> illustrates a call flow for the various nodes functionalities, according to an embodiment. The nodes illustrated in <FIG> correspond to the nodes <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> described with respect to <FIG> above. An example of the call-flow is now described.

The FLUS source control <NUM> sends an indication that the FLUS source control <NUM> is available for streaming to the session orchestrator <NUM> in the form of an HTTP GET (e.g. http://orchasterator ip:<NUM>/GetConfig?token=hmm).

The Session orchestrator <NUM> replies with a configuration file. The file should contain the address of the different controller nodes (e.g., FC1 <NUM>, FC2 <NUM>, FC3 <NUM>) that are participating in controlling the session and are authorized to influence the FLUS Source <NUM>. The session is described in detail after the call-flow.

The FLUS source control <NUM> uses the configuration file to establish connections with all involved controllers:FC1 <NUM>, FC2 <NUM> and FC3 <NUM> (in this example). The connections FC1 to FC3 are used to receive remote control commands (note, there are separate command authorization instructions). The connection can be WebSocket or any other persistent PUSH or <NUM>-way communication mechanism.

The FC-<NUM> session editing controller <NUM> is controlling the FLUS source video capabilities. It checks the FLUS source situation and schedules a session for starting the stream. Such information can include zoom, viewing angle, area covered and duration of the session (any information related to the session directory process). The FC-<NUM> session editing controller <NUM> can start the streaming process by sending a "Start streaming" order to the FLUS source control <NUM>.

The FLUS source controller <NUM> shares its current location (geographical or virtual point of presence) with FC3 <NUM>. FC3 <NUM> analyzes the network situation and suggests a specific bitrate profile.

The FLUS source media <NUM> (part of the FLUS source <NUM>) starts streaming media to the FLUS sink <NUM>.

FC2 <NUM> sends e.g. zoom instructions or position change instruction to the FLUS source <NUM> via websockets.

FLUS source <NUM> shares its status continuously with each of the different FCs <NUM>, <NUM>, <NUM>.

FC1 <NUM> detects that FLUS source <NUM> is leaving an area of service for FC3 <NUM>, and therefore it sends information via websocket (ws) to the FLUS source with the new serving FC3 <NUM>.

FC <NUM><NUM> decides to pause the stream from the FLUS source <NUM> and activate another stream, so it sends a pause command to the source <NUM>.

FC2 <NUM> decides to terminate the session, so it sends a session termination to the FLUS source <NUM>.

The FLUS source <NUM> terminates all (ws) sessions.

Such description is not limited to the mentioned controller, but it can includes different types of controllers depending on the use-case.

An example of the configuration file is now described. The configuration file could be an XML or JSON file which is describing the location, the functionality and access rights (authorization information) for each stream / each remote controller and which protocol it uses.

An example of the configuration file may be as follows:
{[ "control ID" : "FC1", "on/off" : <NUM>, "change location" : <NUM>, "change bitrate" : <NUM>,
"location" : [ "ip/ws_ctrl" ], "connect type" :[ws] "location":http://ip/fc1/]
,["control ID" : "FC2", "on/off" : <NUM>, "change location" : <NUM>, "change bitrate" : <NUM>,
"location" : [ "ip/ws_ctrl" ], "connect type" :[ws] "location":http://ip/fc2/]]}.

<FIG> are additional illustrations of example architectures.

<FIG> illustrates a process <NUM> according to some embodiments. Process <NUM> is a method, performed by a source control entity, for controlling delivery of a media source to a media sink. The method includes the source control entity sending an indication of availability for streaming (step <NUM>); the source control entity establishing connections with one or more controller nodes (step <NUM>); the source control entity receiving a first command from one of the one or more controller nodes (step <NUM>); and the source control entity relaying the first command to the media source (step <NUM>).

In some embodiments, the method further includes the source control entity verifying that the one of the one or more controller nodes is authorized to issue the first command prior to relying the first command to the media source. According to the invention, the one or more controller nodes includes a first controller node authorized to send commands to control a camera associated with the media source and a second controller node authorized to send commands to affect a bitrate associated with the media source. In embodiments, the one or more controller nodes are located remotely from the source control entity.

In some embodiments, the source control entity may be, for example FLUS source control node <NUM>. In some embodiments, the source control entity may be part of a FLUS source that is subdivided into a FLUS source media and the FLUS source control, such as FLUS source <NUM>. In some embodiments, the source control entity sends the indication of availability for streaming to a session orchestrator. In some embodiments, the one or more controller nodes include one or more of an editing controller, a session management controller, and a bitrate controller. In some embodiments, one or more of the one or more controller nodes is remote from the source control entity.

<FIG> is a diagram showing functional modules of node <NUM> (e.g. a FLUS Source Control, F-C) according to some embodiments. As shown in <FIG>, node <NUM> includes a sending unit <NUM>, an establishing unit <NUM>, a receiving unit <NUM>, and a relay unit <NUM>. Sending unit <NUM> is configured to send an indication of availability for streaming; establishing unit <NUM> is configured to establish connections with one or more controller nodes; receiving unit <NUM> is configured to receive a first command from one of the one or more controller nodes; and relay unit <NUM> is configured to relay the first command to the media source.

<FIG> is a block diagram of node <NUM> according to some embodiments. As shown in <FIG>, node <NUM> may comprise: a data processing apparatus (DPA) <NUM>, which may include one or more processors (P) <NUM> (e.g., a general purpose microprocessor and/or one or more other processors, such as an application specific integrated circuit (ASIC), field-programmable gate arrays (FPGAs), and the like); a transmitter <NUM> and a receiver <NUM> coupled to an antenna <NUM> for enabling node <NUM> to transmit data to and receive data from an AN node (e.g., base station); and local storage unit (a. , "data storage system") <NUM>, which may include one or more non-volatile storage devices and/or one or more volatile storage devices (e.g., random access memory (RAM)). In embodiments where node <NUM> includes a general purpose microprocessor, a computer program product (CPP) <NUM> may be provided. CPP <NUM> includes a computer readable medium (CRM) <NUM> storing a computer program (CP) <NUM> comprising computer readable instructions (CRI) <NUM>. CRM <NUM> may be a non-transitory computer readable medium, such as, but not limited, to magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory), and the like. In some embodiments, the CRI <NUM> of computer program <NUM> is configured such that when executed by data processing apparatus <NUM>, the CRI causes node <NUM> to perform steps described above (e.g., steps described above with reference to the flow charts). In other embodiments, node <NUM> may be configured to perform steps described herein without the need for code. That is, for example, data processing apparatus <NUM> may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and/or software.

The here described use-case around "Downlink Distribution of FLUS Content" shows a realization where the FLUS Sink is remote controlling the FLUS Source.

The use case around "Downlink Distribution of FLUS content" combines a FLUS live uplink source with a 3GPP defined distribution system, like PSS or MBMS.

The distribution system is capable of determining the viewership of the session, e.g. using audience measurement mechanisms. For MBMS, the consumption reporting procedure can be leveraged to determine the audience size. There may be other mechanisms to determine the current audience size.

It should be noted, that the application logic, which interacts here with the FLUS Source and the FLUS sink, is not depicted in the reference architecture. The application logic includes use-case and service specific behavior, e.g. is aware about the Distribution System realization. There can be application specific transactions, which are not standardized.

A potential call flow (simplified) is depicted below. The call flow does not further separate the functions on the UE, since the discussion aims at the separation of App specific information and generic information exchange.

Another realization is depicted below. Here, the FLUS Sink is enabled to send remote control instructions like start / stop / pause to the FLUS Source.

The transaction between the Control Function (CTRL) and the FLUS Sink are only around service provisioning, like FLUS source authorization, distribution configuration, etc..

<NUM> and <NUM>: The CTRL function (operations console) configures the drone and the FLUS sink (incl Processing and Distribution). As result, the e2e system is fully provisioned.

<NUM>: A Control Connection between FLUS Source and FLUS Sink is established.

<NUM>: The FLUS Sink triggers the start of the start of the FLUS media.

<NUM>: The FLUS Source in the UE sends the FLUS media data to the FLUS sink. The FLUS sink forwards the media data to the processing function (which generates additional media representations) to the Distribution System (e.g. a DASH Server).

The idea here is to extend FLUS so that a Network Assistance Node (NAssS) can provide information like rate recommendations into the FLUS Source.

Currently, the rate recommendations are only available for 3GP DASH clients, since defined in TS <NUM>.

Observation <NUM>: All three use-case require sending control commands to the FLUS Source (potentially not from the same FLUS Sink).

Observation <NUM>: The Remote Control commands from a Network Assistance function might come from a different node than remote control of camera devices. A different command set may be used for Network Assistance.

It is proposed to include a new remote control interface (The FLUS Sink control the FLUS source) into the FLUS framework for non-IMS based instantiations.

It is proposed to keep rate recommendation and other network assistance functions separate from the FLUS Source remote control. The network assistance may give recommended into to the FLUS source (e.g. bitrate changes, codec re-configuration, etc.), while other functions are authorized to start / stop / pause the FLUS Media or also to influence the capture devices.

It is proposed to update the FLUS reference architecture and include a remote control interface as depicted herein. The new FLUS Remote Control connection (F-RC) is established and maintained by the FLUS Source but used by a FLUS sink for remote control instructions.

It should be possible that a FLUS Source connects to several FLUS RC functions.

Note, whether the remote control commands are defined under F-C umbrella or a new interface name is selected (like F-RC) is for further study. However, current F-C is a FLUS-Sink Provisioning interface and does not contain means so that the FLUS Source establishes a push.

Additionally, while the processes described above and illustrated in the drawings are shown as a sequence of steps, this was done solely for the sake of illustration. Accordingly, it is contemplated that some steps may be added, some steps may be omitted, the order of the steps may be re-arranged, and some steps may be performed in parallel.

Claim 1:
A method, performed by a Framework for Live uplink Streaming, FLUS, source control entity, for controlling delivery of media from
a FLUS media source to a FLUS media sink over a
FLUS control plane, wherein the FLUS source control entity is part of a FLUS media source that is subdivided
into a FLUS source media entity
and the FLUS source control entity, the method comprising
the FLUS source control entity sending an indication of availability for streaming a FLUS media session over a FLUS user plane;
the FLUS source control entity establishing connections with plurality of controller nodes including a first controller node;
the FLUS source control entity receiving a first command from the first controller node targeted to a camera associated with the FLUS media source; and
the FLUS source control entity relaying the first command to the camera;
the FLUS source control entity establishing connections with a second controller node from the plurality of controller nodes, the FLUS source control entity receiving a second command from the second controller node targeted to an encoding process used at the FLUS media
source for the same FLUS media session, the FLUS source control entity relaying the second command to the FLUS media source.