Patent Description:
In iterative data processing where data processing is distributed to multiple clients, such as for example federated learning, data needs to be distributed to the clients in each iteration. To avoid replicate transmissions over various interfaces in a communication system used for the data distribution (this may include internal core network interfaces as well as the core network - radio access network interface and the air interface), multicast may be used for the data transmission. However, depending on the data processing scheme, it may be desirable to not send the data to all clients that are registered for the data processing in each iteration, but to select a subset of the clients for each iteration to which the respective data is transmitted. This may for example be done for privacy but, e.g. in case of federated learning, better training results. So, it may be necessary to establish, for each iteration, a new multicast session for the respective subset. Since this leads to a high signalling overhead, more efficient approaches for distributing data to clients in an iterative data processing are desirable.

The publication <CIT> describes federated learning and the usage of multicast for transmission of downlink traffic.

The publication <NPL> gives a survey of contemporary distributed machine learning techniques such as federated learning.

A method for distributing data to clients in an iterative data processing is provided including establishing a multicast session with a multiplicity of clients in a wireless communication system, wherein the multicast session is provided by means of one or more data replication components for replicating multicast data and, for each of multiple iterations of the iterative data processing, determining, by a server of the wireless communication system, a subset of the multiplicity of clients for the iteration, determining data to be distributed to the determined subset of clients, notifying the one or more data replication components about the determined subset, providing, by the server, the determined data to the one or more data replication components and controlling the one or more data replication components to replicate the data according to the determined subset and to transmit the determined data to each client of the determined subset within the multicast session.

The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details and aspects of this disclosure in which the invention may be practiced. Other aspects may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various aspects of this disclosure are not necessarily mutually exclusive, as some aspects of this disclosure can be combined with one or more other aspects of this disclosure to form new aspects.

In the following, various examples will be described in detail.

Federated learning (FL) is a decentralized machine learning (ML) technique that trains a ML model <NUM> (e.g. a deep neural network DNN) across multiple (FL) clients <NUM> using, for each client, a local training dataset <NUM> under the control of a central server (denoted as FL server) <NUM> (e.g. implemented by a <NUM> or <NUM> cloud).

It typically includes multiple training iterations, wherein in each training iteration (or training "cycle"),.

The resulting version (i.e. new version of the last iteration) of the ML model <NUM> may then be used by a model consumer such as the NWDAF(AnLF) (Network Data Analytics Function (Analytics Logical Function)) which may use it to make predictions from data obtained from an environment (such as communication resource need).

Dynamic member selection may be used to:.

With regard to realization in the framework of a <NUM> communication network, the FL server <NUM> may be an application function and the clients <NUM> are applications in UEs (User Equipments).

Federated Learning is a decentralized application such that there is considerable communication between the FL server <NUM> and the FL clients <NUM>. Accordingly, communication efficiency specifically in the case of a large number of clients is desirable. It should be noted that in each iteration (e.g., cycle of FL training) the same data (i.e., the current version of the (global version of the) ML model <NUM>) is distributed from the server <NUM> to multiple clients <NUM>, i.e. there may be redundant data transmission over the respective communication network that is used for the FL data transmission. In the following, a FL is assumed which uses dynamic member selection, i.e. the FL server <NUM> selects a subset of the clients <NUM> for each iteration (i.e. there is a "dynamic selection" of clients).

For transmitting the same data to multiple devices, communication networks typically have a multicast functionality.

<FIG> shows a radio communication system <NUM>, in this example configured according to <NUM> (Fifth Generation) as specified by 3GPP (Third Generation Partnership Project), with respect to multicast, in particular the various interfaces involved.

The radio communication system <NUM> includes a mobile radio terminal device <NUM> such as a UE (user equipment), a nano equipment (NE), and the like. The mobile radio terminal device <NUM>, also referred to as subscriber terminal, forms the terminal side while the other components of the radio communication system <NUM> described in the following are part of the mobile radio communication network side, i.e. part of a mobile radio communication network (e.g. a Public Land Mobile Network PLMN).

Furthermore, the radio communication system <NUM> includes a radio access network <NUM>, which may include a plurality of radio access network nodes, i.e. base stations configured to provide radio access in accordance with a <NUM> (Fifth Generation) radio access technology (<NUM> New Radio). It should be noted that the radio communication system <NUM> may also be configured in accordance with LTE (Long Term Evolution) or another mobile radio communication standard but <NUM> is herein used as an example. Each radio access network node may provide a radio communication with the mobile radio terminal device <NUM> over an air interface. It should be noted that the radio access network <NUM> may include any number of radio access network nodes.

The radio communication system <NUM> further includes a core network (CN, here 5GC) including an Access and Mobility Management Function (AMF) <NUM> connected to the RAN <NUM> and a Unified Data Management (UDM) <NUM>. Here and in the following examples, the UDM may further consist of the actual UE's subscription database, which is known as, for example, the UDR (Unified Data Repository). The core network one or more PCFs (Policy Control Functions) <NUM>.

The core network further includes a Session Management Function (SMF) <NUM> and multiple a User Plane Functions (UPF) <NUM>. The SMF <NUM> is for handling PDU (Protocol Data Unit) sessions, i.e. for creating, updating and removing PDU sessions and managing session context with the User Plane Function (UPF).

The core network further includes an application function (AF) <NUM> (or application server AS). The AF <NUM> is connected to other core network components such as the PCF <NUM> via an NEF (Network Exposure Function), in particular in case the AF <NUM> is maintained by a third party (i.e. a party other than the operator of the mobile radio communication system <NUM>).

To support the 3GPP MBS (Multicast/Broadcast Service), there are MB (Multicast Broadcast) versions of the SMF <NUM> and the UPF <NUM>: an MB-SMF <NUM> and an MB-UPF <NUM>, respectively. Further, the communication system includes an MBSF <NUM> (Multicast/Broadcast Service Function) and MBSTF <NUM> (MBSTF Multicast/Broadcast Service Transport Function.

MBS data (i.e. "traffic"), e.g. from the FL <NUM>, is received by the communication system's core network <NUM>. The MBS data should be transmitted to a plurality of UEs <NUM>, <NUM> via the communication system's radio access network (RAN) <NUM>. The core network <NUM> may use separate PDU sessions <NUM> for multiple UEs <NUM> or may use a shared transport <NUM> to each base station of the RAN <NUM> for multiple UEs <NUM> (served by the base station to which the base station then transmits the data using radio point-to-point or radio point-to-multipoint transmission). These two options are referred to transmission via 5GC individual MBS traffic delivery and transmission via 5GC shared MBS traffic delivery, respectively. In any case (since it can be assumed that not all clients <NUM> are served by the same base station), there is a data replication <NUM> at one or more points in the communication system, i.e. the communication system includes are one or more data replication components which replicate the data such that there is a version of the data for each UE (or at least each for each base station (or radio cell): within a radio cell, it is possible that data is only transmitted once to multiple UEs).

In context of a multicast functionality such as 3GPP's MBS, there are typically the following concepts:.

It should be noted that in the following examples, Federated Learning is used as use case but embodiments may also be applied to other applications that iteratively transfer the same data to a dynamic group of UEs, i.e. for other types of iterative data processing (such as distributed graph processing or MapReduce applications).

<FIG> shows a flow diagram <NUM> illustrating traffic in a more general application example.

A server <NUM> and multiple clients <NUM> (numbered <NUM> to N) are involved in the flow.

In each iteration <NUM>, the same data (e.g. in case of FL the current version of Global Model (GM-i)) is sent from <NUM> server to (a subset of) the clients (where i is the iteration index). So, in each iteration data can be multicast.

In each iteration <NUM>, a subset of the clients <NUM> needs to receive the data. According to various embodiments, there is therefore a target client (e.g. target UE) filtering in each iteration <NUM> in the multicasting (e.g. a filtering of UEs in a multicast group which contains all the clients <NUM>).

The overall data transmission is not a multimedia stream but includes periodic data transmissions if the size of data in all iterations is the same, i.e. size(GM-i) = size(GM-j). So, the data volume (per iteration) can be shared with 5GC to derive QoS (Quality of Service) parameters like Maximum Data Burst Volume.

In the following, embodiments will be described in context of federated learning.

<FIG> shows a flow diagram <NUM> illustrating an FL procedure, in particular with respect to data transmission using multicast via a filtering mechanism provided according to various embodiments.

An FL server <NUM> (e.g. corresponding to FL server <NUM>) and FL clients <NUM> (e.g. UEs, numbered <NUM> to N, e.g. corresponding to FL clients <NUM>) are involved in the flow. Communication between the FL server <NUM> and the FL clients <NUM> is performed, in this example, via a <NUM> communication system, in particular an MF-SMF <NUM>, an MB-UPF <NUM> and a data replication component <NUM> which may be a UPF, an MB-UPF or a RAN (see <FIG> as well as <FIG> which illustrates the two options where data replication takes place).

As mentioned above, according to various embodiments, data is filtered in a multicast session, i.e. in each training iteration, data is transmitted to the client subset selected (or defined) for that training iteration. Accordingly, in <NUM>, the FL server <NUM> establishes a "filtered" multicast session (i.e. a multicast session which has this filtering feature) by means of the communication system's core network.

The Filtered Multicast Indicator (i.e. an indicator which indicates that the multicast session is a filtered multicast session) may for example be included in the Nmbsmf_MBSSession_CreateRequest message sent from NEF or MBSF to MB-SMF (e.g. according to 3GPP TS <NUM> version <NUM>. <NUM> Release <NUM> clause <NUM>. <NUM>, Figure <NUM>. <NUM>-<NUM>). The data volume of periodic traffic (i.e. size of data for the global model) may be defined as part of the MBS service information in the Nmbsmf_MBSSession_CreateRequest message.

In <NUM>, the FL clients <NUM> register with the FL server <NUM> (for participation in the training) and the FL server <NUM> initializes the FL clients <NUM> for the training, including informing them about the identification of the multicast session (in this example an MBS Session ID). In <NUM>, the FL clients <NUM> join the multicast session.

The training loop is then performed, which includes, in each iteration <NUM>.

The FL server <NUM> (corresponding to an AF <NUM>) may share, in each iteration, a list of UEs of the respective FL client subset (i.e. list of target UEs) with the replicating entities (i.e. replication components), e.g. MB-UPF, UP, RAN (specifically base stations, e.g. gNBs) either before or while transmitting the multicast data (i.e. for example in or before the steps of multicast data transmission in TS <NUM>, clause <NUM>. <NUM>, Figure <NUM>. <NUM>-<NUM>, see <FIG> steps <NUM>, <NUM> and <NUM>).

<FIG> shows a flow diagram <NUM> where the FL server indicates the FL client subset by means of a target UE list in user plane to the replication components (found by control plane signalling).

A UE <NUM> (representing the FL clients), a RAN <NUM>, an AMF <NUM>, an SMF <NUM>, a UPF <NUM>, an NRF <NUM>, a PCF <NUM>, an MB-SMF <NUM>, and MB-UPF <NUM> and an AF <NUM> (corresponding to the FL server) are involved in the flow.

In <NUM> to <NUM>, UE <NUM> joins a multicast session.

In <NUM>, the AF <NUM> provides multicast data (e.g. the specification of a current version of the ML model to be trained). With the multicast data, the AF <NUM> provides a target UE list (i.e. a list of UEs corresponding to the FL clients selected for the current iteration).

As explained with reference to <FIG>, there are two options:.

In the first option, in <NUM>, the core network control plane determines the target base stations (i.e. the base stations serving the target UEs). In <NUM>, the MB-UPF <NUM> transmits the multicast data as well as the target UE list to the determined base stations of the RAN <NUM> which transmit the data to each target UE <NUM> (e.g. using an MBS Radio Bearer) in <NUM>.

In the second option, in <NUM>, the core network control plane determines the target UPFs (i.e. the UPFs serving the target UEs). In <NUM>, the MB-UPF <NUM> transmits the multicast data as well as the target UE list to the determined UPFs which transmit the data via the RAN <NUM> to each target UE <NUM> in <NUM> using a PDU session per UE <NUM>.

<FIG> shows a flow diagram <NUM> where the FL server indicates the FL client subset before transmitting the multicast data to the replication components (via control plane signalling). The multicast data is associated to the list either implicitly via a time slot in which it is sent (which is associated with the list) or explicitly via a ID tag (which is associated with the list) included in the user plane transmission (i.e. together with the multicast data).

As in <FIG>, a UE <NUM> (representing the FL clients), a RAN <NUM>, an AMF <NUM>, an SMF <NUM>, a UPF <NUM>, an NRF <NUM>, a PCF <NUM>, an MB-SMF <NUM>, and MB-UPF <NUM> and an AF <NUM> (corresponding to the FL server) are involved in the flow.

In <NUM>, the AF <NUM> transmits the target UE list for the current iteration as well as the indication of a time slot or ID tag associated with the target UE list to the MB-SMF <NUM>.

Like in <FIG>, there is a distinction of the two options:.

For the first option, in <NUM>, the core network control plane determines the target base stations (i.e. the base stations serving the target UEs). The MB-SMF <NUM> then informs the MB-UPF about the target base stations in <NUM> and transmits the target UE list and the associated time slot or ID tag, respectively, to the target base stations in <NUM>.

For the second option, in <NUM>, the core network control plane determines the target UPFs (i.e. the UPFs serving the target UEs). The MB-SMF <NUM> then informs the MB-UPF about the target UPFs in <NUM> and transmits the target UE list and the associated time slot or ID tag, respectively, to the target UPFs in <NUM>.

In <NUM>, the AF <NUM> provides multicast data (e.g. the specification of a current version of the ML model to be trained). The AF <NUM> does this in a certain time slot associated with the target UE list or provides, with the multicast data, the ID tag of the target UE list (the latter case is illustrated in <FIG>).

For the first option, in <NUM>, the MB-UPF <NUM> forwards the multicast data to the target base stations (possibly indicating the ID tag of the target UE list) which transmit the data to each target UE <NUM> (e.g. using an MBS Radio Bearer) in <NUM>.

In the second option, in <NUM>, the MB-UPF <NUM> forwards the multicast data to the target UPFs (possibly indicating the ID tag of the target UE list) which transmit the data via the RAN <NUM> to each target UE <NUM> in <NUM> using a PDU session per UE <NUM>.

In the following, examples for the transmission of the target UE list (as in the example of <FIG>) are described with reference to <FIG> and <FIG> for the two possible options mentioned above.

<FIG> illustrates the transmission of the target UE list in an extension header (e.g. of GTP-U or IPv6) from the AF <NUM> via the MB-UPF <NUM> to the target base station <NUM> serving the target UEs <NUM> ("UE-<NUM>" and "UE-<NUM>" in this example) for the case of shared MBS traffic delivery (option <NUM> above).

The MB-UPF <NUM> may determine the target base station <NUM> by consulting the MB-SMF <NUM> which in turn may consult the AMF <NUM>.

<FIG> illustrates the transmission of the target UE list in an extension header (e.g. of GTP-U or IPv6) from the AF <NUM> via the MB-UPF <NUM> to the target UPF <NUM> serving the target UEs <NUM> ("UE-<NUM>", "UE-<NUM>" and "UE-<NUM>" in this example) for the case of individual MBS traffic delivery (option <NUM> above).

In the following, examples for the usage of a time slot to specify the target UEs (see description of <FIG>), also referred to as "CP assisted" case, are described with reference to <FIG> and <FIG> for the two possible options mentioned above.

<FIG> illustrates the transmission of multicast data for the case of shared MBS traffic delivery in the CP assisted case.

The AF <NUM> first transmits (via control plane) the information to the MB-SMF <NUM> that a certain set of UEs ("UE-<NUM>" and "UE-<NUM>" in this example), is associated with a certain time slot. The time slot implicitly thus associates multicast data (that is sent in the time slot) with the target UE list.

After this has been done, the AF <NUM> may transmit multicast data in the respective time slot to the MB-UPF <NUM>. The MB-UPF <NUM> is informed about the target base station <NUM> by the MB-SMF <NUM> which determines (for the time slot) the target UE list and consults the AMF <NUM> for the target base station <NUM>. The MB-UPF <NUM> may then send the multicast data to the target base station <NUM> serving the target UEs <NUM> during the given time slot. The MB-SMF <NUM> may also inform the RAN about the association of target UEs <NUM> with the time slot. It should be noted that the complete control plane signalling (shown hatched) may be completed before the user plane signalling (i.e. the transmission of multicast data).

<FIG> illustrates the transmission of multicast data for the case of individual MBS traffic delivery in the CP assisted case.

The AF <NUM> first transmits (via control plane) the information via the MB-SMF <NUM> to each target UPF <NUM> that a certain set of UEs <NUM> ("UE-<NUM>", "UE-<NUM>" and "UE-<NUM>" in this example) served by the UPF is associated with a certain time slot. The time slot implicitly thus associates multicast data (that is sent in the time slot) with the target UEs served by that UPF <NUM>.

After this has been done, the AF <NUM> may transmit multicast data in the respective time slot to the MB-UPF <NUM> which forwards it to each target UPF <NUM>. The target UPF <NUM> then transmits the multicast data to the target UEs <NUM> (using individual PDF sessions) during the time slot. Again, it should be noted that the complete control plane signalling (shown hatched) may be completed before the user plane signalling (i.e. the transmission of multicast data).

In the following, examples for the usage of an ID tag to specify the target UEs (see description of <FIG>), also referred to as "CP-UP combined" case, are described with reference to <FIG> and <FIG> for the two possible options mentioned above.

<FIG> illustrates the transmission of multicast data for the case of shared MBS traffic delivery in the CP-UP combined case.

The AF <NUM> first transmits (via control plane) the information to the MB-SMF <NUM> that a certain set of UEs ("UE-<NUM>" and "UE-<NUM>" in this example), is associated with a ID tag.

After this has been done, the AF <NUM> may transmit multicast data including the tag ID to the MB-UPF <NUM>. The tag ID is for example carried in the extension header (GTP-U, IPv4, IPv6) of the multicast data and explicitly associates multicast data with the target UE list. The MB-UPF <NUM> is informed about the target base station <NUM> by the MB-SMF <NUM> which determines (from the tag ID) the target UE list and consults the AMF <NUM> for the target base station <NUM>. The MB-UPF <NUM> may then send the multicast data to the target base station <NUM> serving the target UEs <NUM>. The MB-SMF <NUM> may also inform the RAN about the association of target UEs <NUM> with the tag ID. It should be noted that the complete control plane signalling (shown hatched) may be completed before the user plane signalling (i.e. the transmission of multicast data).

<FIG> illustrates the transmission of multicast data for the case of individual MBS traffic delivery in the CP-UP combined case.

The AF <NUM> first transmits (via control plane) the information via the MB-SMF <NUM> to each target UPF <NUM> that a certain set of UEs <NUM> ("UE-<NUM>", "UE-<NUM>" and "UE-<NUM>" in this example) served by the UPF is associated with a certain ID tag.

After this has been done, the AF <NUM> may transmit multicast data including the tag ID to the MB-UPF <NUM> which forwards it to each target UPF <NUM>. The tag ID is for example carried in the extension header (GTP-U, IPv4, IPv6) of the multicast data and explicitly associates multicast data with the target UE list. The target UPF <NUM> then transmits the multicast data to the target UEs <NUM> (using individual PDF sessions). Again, it should be noted that the complete control plane signalling (shown hatched) may be completed before the user plane signalling (i.e. the transmission of multicast data).

For communication between MB-UPF and MB-SMF, the N4 (specifically the N4mb interface) may be extended by a PFCP message type to query location of a UE (to determine its serving base station).

Regarding the communication between AF and MB-SMF (in particular to specify the current selected subset), an MB-SMF service extension may be introduced, e.g. the Nmbsmf_MBSSession_ContextUpdate message may be extended such that it allows to. pause or resume of multicast data reception.

Further, the N4 (specifically the N4mb interface) may be extended with an extension to update the FAR IE (Forwarding Action Rule Information Element) within the PFCP Session Modification Request to include only the current target UE list.

<FIG> illustrates a multicast of data from an AF <NUM> via MB-UPFs <NUM> and base stations <NUM> to UEs <NUM>.

Accordingly, the multicast data travels over the N6 (specifically N6mb) interface and the N3 (specifically N3mb interface) as well as over dedicated radio bearers.

The approaches described herein allow using a multicast session to distribute global model from FL server to FL clients such that the number of transmissions of multiple replicas of the global model from AF to the core network (e.g. 5GC) is reduced. This includes transmission and resource consumption efficiency (e.g. in terms of number of packets) on the N6 interface and on the N3 interface if the shared MBS traffic delivery is deployed.

The usage of target UE list for the multicast data eliminates unnecessary replication of data for the clients that have not been selected for this iteration of FL training, thus including transmission and resource consumption efficiency and eliminating MBS session creation and join procedure per FL cycle (thus increasing signalling efficiency).

The transmission and resource consumption efficiency that can be achieved is especially high if a small number of (MB-)UPFs and base stations server the selected clients, UEs do not handover during an iteration.

In summary, according to various embodiments, a method is provided as illustrated in <FIG>.

<FIG> shows a flow diagram <NUM> illustrating a method for distributing data to clients in an iterative data processing.

In <NUM>, a multicast session (also denoted as "filtered" multicast session) with a multiplicity of clients is established, wherein the multicast session is provided by means of one or more data replication components for replicating multicast data.

In <NUM>, for each of multiple iterations of the iterative data processing,.

It should be noted that the steps of <FIG> do not necessarily be performed in the illustrated order (e.g. the one or more data replication components are notified about the determined subset before the data is determined or the data may be determined before the subset is determined).

According to various embodiments, in other words, an approach (on architectural level) for efficiently distributing (exchanging) data to clients is provided. This is achieved by using the same multicast session (over multiple iterations, i.e. avoiding establishing a multicast session for each determined subset of clients) but controlling, within the multicast session, the subset of clients to which data is sent. So, the data is sent for multiple operations within the same multicast context. In other words, each iteration has an associated (determined) data set to be sent in the iteration and the data sets of multiple iterations are sent within the same multicast session (i.e. the same multicast (session) context).

The clients may for example correspond to mobile terminals in a mobile communication system (e.g. each client may correspond to a respective mobile terminal).

Transmitting the data within the same multicast context (i.e. keeping the multicast context over multiple iterations) may for example include.

It should be noted that at least for some iterations, the subset changes from iteration to iteration. The size of data may be the same for all iterations.

As described above, the one or more data replication components may be notify about the respective subset via id tag, time slot, control plane or user plane, i.e. the data replicating entities may be instructed to apply target client (e.g. UE) list filtering by.

According to various embodiments, a communication system arrangement (e.g. a server or server system) is provided configured to perform the method of <FIG>.

The communication system arrangement includes components such as one or more communication interfaces (e.g. for communication with core network components, radio access network components and/or mobile terminals), one or more processors and memories.

The components of the communication system arrangement may for example be implemented by one or more circuits. A "circuit" may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof. Thus a "circuit" may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g. a microprocessor. A "circuit" may also be a processor executing software, e.g. any kind of computer program. Any other kind of implementation of the respective functions described above may also be understood as a "circuit".

Claim 1:
A method for distributing data to clients (<NUM>, <NUM>) in an iterative data processing, comprising:
establishing (<NUM>) a multicast session with a multiplicity of clients (<NUM>, <NUM>) in a radio communication system (<NUM>), wherein the multicast session is provided by means of one or more data replication components for replicating multicast data; and
for each of multiple iterations of the iterative data processing,
determining (<NUM>), by a server (<NUM>, <NUM>) of the radio communication system (<NUM>), a subset of the multiplicity of clients (<NUM>, <NUM>) for the iteration;
determining (<NUM>) data to be distributed to the determined subset of clients (<NUM>, <NUM>);
notifying (<NUM>) the one or more data replication components about the determined subset; and
providing (<NUM>), by the server (<NUM>, <NUM>), the determined data to the one or more data replication components and controlling the one or more data replication components to replicate the data according to the determined subset and to transmit the determined data to each client (<NUM>, <NUM>) of the determined subset within the multicast session.