Patent Description:
In <NUM> mobile communications networks, the mobility management entity is the node that communicates with the radio access node to perform mobility related functions on behalf of mobile subscribers. In <NUM> communications networks, the AMF is the node that communicates with the radio access nodes to perform the mobility related functions on behalf of use. The connection between the radio access node and the MME node is referred to as an S1 connection, as the interface between the radio access nodes and the MME is the S1 interface. Similarly, in <NUM> communications networks, the interface between the radio access node and the AMF is referred to as the N2 interface and the connections on the interface are referred to as N2 connections.

One problem that occurs in mobile communications networks, is that MME and AMF nodes may be required to be scaled up or down, especially in cloud configurations. During the scaling up or scaling down of the MME or AMF nodes, S1 or N2 connections may need to be added or removed. Adding or removing S1 or N2 connections may be disruptive of radio access nodes.

In light of these difficulties, there exists a need for improved methods, systems, and computer readable media for allowing reconfiguration of MME or AMF nodes without adversely affecting the S1 or N2 connections with the radio access nodes.

In <CIT>, a network element having a Non-Access Stratum Node Selection Function for discriminating information is disclosed.

In <CIT>, a method for distributing Sigtran connections among signal transfer point message processors is disclosed.

A method for distributing connections to mobility management node instances is set forth in claim <NUM>. In the subject matter described herein, the IP address published by the connection load balancer is associated with a loopback interface of the mobility management node instances. Associating the IP address with a loopback address of the mobility management node instances enables the mobility management node instances to use the IP address as an alias without broadcasting address resolution protocol (ARP) or ICMP messages containing the IP address.

The subject matter described herein may be implemented in hardware, software, firmware, or any combination thereof. As such, the terms "function" "node" or "module" as used herein refer to hardware, which may also include software and/or firmware components, for implementing the feature being described. In one exemplary implementation, the subject matter described herein may be implemented using a computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps. Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.

Mobile networks are growing with multiple technologies and interconnects to carry the signaling. One current challenge with mobile communications networks is moving applications and nodes that provide services in mobile communications networks into the cloud. Cloud based environments also require smart scaling up and down of networks. There is potential need to reconfigure the MME/AMF nodes in case they are scaling up / down in the cloud. All of the S1 / N2 connections need to be added/removed while scaling up and down. S1 connection are connections between radio access network (RAN) nodes and the MME. N2 connections are connections between RAN nodes and the AMF.

The subject matter described herein aims at resolving the problem of re-configuration of the MME/AMF nodes using a connection load balancer (CLB), which may be located in the cloud. This will allow the networks to manage the scaling up and down of MMEs and/or AMFs without affecting the RAN nodes connected to the MME/AMF nodes in the cloud or on premise.

In current systems, multiple MME/AMF nodes operate as multiple instances to ease scaling / high-availability processing of S1/N2 traffic from the RAN side, such that each MME/AMF instance has dedicated connections with external RAN nodes. Each MME/AMF instance may have multiple S1/N2 connections from same RAN node. One RAN node may have S1/N2 connections spanning across multiple MME/AMF instances for high availability.

When any one or more MME/AMF instances goes down (due to an error or scaling down), all of the S1/N2 connections with these instances are disconnected. When an MME/AMF instance comes up (after disaster recovery or scale up), new S1/N2 connections have to be configured from the RAN. Because of the above facts, the remote RAN node seeking to reconnect with the MME/AMF instance needs to manage each S1/N2 connection with different IP addresses to achieve load distribution across MME/AMF instances. Requiring the remote RAN separately manage each S1/N2 connection with a different IP address causes a configuration burden on the remote RAN node, especially in the case where connections need to be reestablished after an MME or AMF becomes operational again after a period of being non-operational.

<FIG> illustrates a RAN node connected to multiple MME/AMF instances where the MME/AMF instances represent a single MME/AMF node. Referring to <FIG>, RAN node <NUM> is connected to multiple MME/AMF instances <NUM> via S1/N2 connections <NUM>. Each MME/AMF instance <NUM> includes Ethernet interfaces <NUM> on networks A and B. Each Ethernet interface <NUM> includes a MAC address, and a separate IP address is associated with each Ethernet interface <NUM>.

In the illustrated example, the S1/N2 connections <NUM> each include the primary ingress connection path <NUM> associated with one of the Ethernet interfaces of MME/AMF <NUM> and a secondary ingress connection path <NUM> associated with another Ethernet interface <NUM> of an MME/AMF instance <NUM>. The primary and secondary communications path may be stream control transmission protocol (SCTP)-based multi-homed connection paths. However, the subject matter described herein is not limited to using SCTP multi-homed connections with only two associations. The number of associations can scale from <NUM> to a maximum number of associations supported by the SCTP protocol. However, the subject matter described herein is not limited to using SCTP as the transport layer protocol. Transmission control protocol (TCP) could be used instead of SCTP without departing from the scope of the subject matter described herein.

In the illustrated example, the IP address IP-Node-N-A is the IP address of MME/AMF instance-<NUM> <NUM> on signaling device or Ethernet interface A <NUM>. The IP address IP-Node-N-B is the IP address of MME/AMF instance-<NUM> <NUM> on signaling device B or Ethernet interface B <NUM>. The IP address IP-Node-N-A is the IP address of MME/AMF instance-N <NUM> on signaling device or Ethernet interface A <NUM>. The IP address IP-Node-N-B is the IP address of MME/AMF instance-N <NUM> on signaling device or Ethernet interface B <NUM>.

Remote RAN nodes can connect to MME/AMF instances over S1/N2 connections where the MME/AMF instances are implemented on a single processor or on multiple processors, depending on the redundancy model chosen. The subject matter described herein is not limited to managing S1/N2 connections to MME/AMF instances implemented on a single process or on multiple processors. Either case is intended to be within the scope of the subject matter described herein.

<FIG> illustrates some of the issues with the current network architecture when an MME/AMF instance goes down. When an MME/AMF instance goes down, the RAN node formerly connected to the affected MME/AMF may need to reconfigure the affected connections with fixed IPs of another MME/AMF instance. Such reconfiguration is burdensome for both ends of the S1/N2 connection.

In addition to the reconfiguration issues caused by the current architecture, automatic connection distribution to the least loaded MME/AMF instance may not be possible. RAN nodes may not have visibility into the loading of MME/AMF instances, especially when the instances change due to outages or changes in cloud configurations. As a result, distribution of S1/N2 connections by the RAN nodes may be sub-optimal.

Another issue that may be present with the current network architecture is that when any MME/AMF instance (or multiple instances) go down, connections connected to it/them are not automatically distributed to other MME/AMF instances, thus leaving fewer operational connections unless manual intervention is performed to re-configure connections with other MME/AMF instances. When an MME/AMF instance goes out of service automatic reconfiguration of connections may not be possible.

In the example illustrated in <FIG>, when one of MME/AMF instances <NUM> goes down, the S1/N2 connections connected to the failed or out of service MME/AMF instance <NUM> are dropped. Reconnection attempts with the failed MME/AMF <NUM> will result in connection failure. In addition, there is no automatic load distribution to the remaining MME/AMF <NUM>.

To avoid or mitigate at least some of the difficulties associated with the current S1/N2 connection architecture, a connection load balancer may be provided between the RAN node(s) and the MME/AMF instances to perform automatic reconfiguration of connections when the MME/AMF instance configuration changes without requiring reconfiguration of connections between the connection load balancer and the remote RAN nodes. For example, the connection load balancer could be used to resolve the issues mentioned with respect to <FIG>.

In one implementation, the connection load balancer exposes a single set of public IP addresses to connect to all the MME/AMF instances being used to provide service to the RAN nodes. This provides the flexibility to the MME/AMF node to scale up or down without requiring a change in the public IP address information provided to the RAN nodes. Because the public IP addresses exposed by the connection load balancer are independent from the MME/AMF configuration, the MME/AMF configuration can change without requiring the remote RAN nodes to reconfigure S1/N2 connections.

In addition to reducing the amount of reconfiguration required on the part of remote RAN nodes, the connection load balancer provides the flexibility to automatically balance the connections on the available MME/AMF resources, hence optimizing the resource usage. For example, the connection load balancer may maintain measurements of S1/N2 connection loading of the MME/AMF instances and use these measurements to make S1/N2 connection distribution decisions. In the case of a message processing capacity loss in the MME/AMF node the connection load balancer can redistribute the connections to other MME/AMF instances and provide the same level of service to the remote RAN nodes.

In <FIG>, connection load balancer <NUM> includes Ethernet interfaces <NUM> and <NUM>. Connection load balancer <NUM> only exposes the IP addresses IP-Node-A and the IP address IP-Node-B to RAN devices, such as RAN node <NUM>. The IP addresses IP-Node-<NUM>-A, IP-Node-<NUM>-B, IP-Node-N-A, and IP-Node-N-B are not published to the RAN nodes. As virtual entities, connection load balancer <NUM> and/or mobility management node instances <NUM> may represent a virtual machine that runs on a hypervisor on cloud computing hardware.

<FIG> illustrates one example of a proposed solution using the connection load balancer for ingress traffic. In <FIG>, connection load balancer <NUM> receives S1/N2 connection requests from RAN node <NUM>, performs load balancing to load balance the requests among MME/AMF instances <NUM>, and forwards the connection requests to the selected MME/AMF <NUM>. Because the IP addresses of MME/AMF <NUM> were not exposed to RAN node <NUM>, if MME/AMF instances <NUM> need to be scaled up or down, such scaling can be handled seamlessly at connection load balancer <NUM>, as will be described in more detail below. The IP address IP-Node-A is published to RAN node <NUM> and other RAN nodes on network A. RAN node <NUM> and the other RAN nodes on network A use the IP address IP-Node-A as the destination IP address for ingress S1/N2 connections on network A, where the ingress direction is from the RAN nodes to the MME/AMF instances. The IP address IP-Node-B is published to RAN node <NUM> and other RAN nodes on network B. RAN node <NUM> and the other nodes on network B use the IP address IP-Node-B as the destination IP address for ingress S1/N2 connections on network B.

<FIG> illustrates the path of egress traffic using a network that includes connection load balancer <NUM>. In <FIG>, it can be seen that egress traffic travels from MME/AMF instances <NUM> to RAN node <NUM> without passing through connection load balancer <NUM>. The egress direction, in this example, refers to the direction from the MME/AMF instances to the RAN nodes. Egress traffic from the MME/AMF instances will not pass through connection load balancer <NUM> because the egress traffic is addressed to the IP addresses of the remote RAN nodes obtained from ingress messages from the remote RAN nodes.

<FIG> illustrate an initialization stage of connection load balancer <NUM>. In <FIG> step A. <NUM>, a border gateway <NUM>, which may be connected between RAN node <NUM> and connection load balancer <NUM>, receives broadcast gratuitous address resolution protocol (ARP) messages or ICMPv6 neighbor discovery messages from connection load balancer <NUM>. The broadcast gratuitous ARP messages or ICMPv6 neighbor discovery messages associate the IP addresses IP-Node-A and IP-Node-B with media access control (MAC) addresses MAC-A and MAC-B, respectively. <NUM>, border gateway <NUM> updates its MAC table to associate the IP addresses IP-Node-A and IP-Node-B with MAC addresses MAC-A and MAC-B respectively. It should be noted that the MAC addresses MAC-A and MAC-B are associated with Ethernet interfaces <NUM> and <NUM> of connection load balancer <NUM>, rather than of the MME/AMF instances.

<FIG> illustrates steps for configuring the MME/AMF instances. Referring to <FIG>, in step B. <NUM>, each MME/AMF instance configures alias IP addresses IP-Node-A and IP-Node-B on its loopback interface. Since the loopback interface does not have a MAC address, no ARP messages and ICMPv6 neighbor discovery messages are sent by these MME/AMF interfaces. <NUM>, each MME/AMF instance is configured with a multi-homed responder connection that begins listening for incoming S1/N2 connection requests on the address IP-Node-A for the primary path and the address IP-Node-B for the secondary path. As indicated above, the IP addresses IP-Node-A and IP-Node-B are associated with the MAC addresses MAC-A and MAC-B of connection load balancer <NUM>. Because remote RAN nodes use the IP addresses IP-Node-A and IP-Node-B to connect with plural different MME/AMF instances, and the connection load balancer decides which MME/AMF instance should handle a given connection request, MME/AMF instances can be reconfigured without requiring remote RAN nodes to reconfigure their S1/N2 connections as a result of the reconfiguration.

As stated above, one function that may be performed by connection load balancer <NUM> is distribution of S1/N2 connections from remote RAN nodes among MME/AMF instances. <FIG> is a message flow diagram illustrating an example of messages that may be exchanged in a connection distribution algorithm implemented by connection load balancer <NUM>. Referring to <FIG>, in step C. <NUM>, an S1/N2 connection request for Connection-<NUM> with destination IP addresses {IP-Node-A, IPNode-B} and source IP addresses {R-IP-x, R-IP-y} and destination port P1 arrives from a remote RAN node.

<NUM>, border gateway <NUM> queries its MAC table for {IP-Node-A, IP-Node-B} and forwards the connection request for Connection-<NUM> to connection load balancer <NUM>. <NUM>, connection load balancer <NUM> applies a connection distribution algorithm to determine the MME/AMF instance to which the connection request should be forwarded. In one exemplary implementation, connection load balancing is performed based on the following parameters:.

Once connection load balancer <NUM> identifies the MME/AMF instance to which the connection request should be forwarded, connection load balancer <NUM> may perform the following steps:.

<NUM>, connection load balancer <NUM> forwards the connection request to MME/AMF instance-i as decided in step C.

<NUM>, MME/AMF instance-i accepts the connection. Responses are sent directly to the remote peer via a border gateway. The connection load balancer is not in the return path for messages sent from MME/AMF instance-i to RAN node <NUM>.

<FIG> is a message flow diagram illustrating signaling for call continuity in a network that includes connection load balancer <NUM>. Referring to <FIG>, in step D. <NUM>, S1/N2 message arrives on connection-<NUM> with destination IP addresses {IP-Node-A, IP-NODE-A}, source IP addresses {RIP-x, R-IP-y} and destination port P1.

<NUM>, border gateway <NUM> queries its MAC table for {IP-Node-A, IP-Node-B} and forwards the S1/N2 message on connection-<NUM> to connection load balancer <NUM>. In this example, it is assumed that the connection request for Connection-<NUM> has already been processed as described above.

<NUM>, connection load balancer <NUM> queries its association database (in cache memory) to determine the MME/AMF instance to which connection load balancer <NUM> previously forwarded the connection request message. In this example, it is assumed that the following association record exists in the association database maintained by connection load balancer <NUM>:
{R-IP-x, P1, MME/AMF Instance-i)} → MME/AMF Instance-i. This entry was created in by connection load balancer <NUM> in its association database based on the application of the load balancing algorithm to the connection request, as described above.

<NUM>, connection load balancer <NUM> forwards the message to MME/AMF Instance-i as determined in step D.

<NUM>, MME/AMF Instance-i accepts the message, process the message, prepares the response and sends the response over same connection. Responses are sent directly to the remote peer via border gateway <NUM>. Connection load balancer <NUM> is not in the forwarding path for messages sent from MME/AMF instances <NUM> to RAN nodes <NUM>.

<FIG> is a diagram illustrating RAN connection reestablishment. In <FIG>, in step <NUM>, MME/AMF instance <NUM> <NUM> serving S1/N2 connection <NUM> goes down. In step <NUM>, S1/N2 connection <NUM> connected with MME/AMF instance <NUM><NUM> drops. In step <NUM>, S1/N2 connection <NUM> reestablishment is attempted by remote RAN node <NUM>. Connection load balancer <NUM> reroutes the connection request to MME/AMF instance-i based on the connection distribution algorithm described above. It is not necessary for remote RAN node <NUM> to know the IP address of the new MME/AMF instance to which the connection is being re-rerouted. Remote RAN node <NUM> sends the reconnection request to the alias IP address published by connection load balancer <NUM> on behalf of the MME/AMF instance, and connection load balancer <NUM> uses its connection distribution algorithm to select an MME/AMF instance for handling the connection re-establishment request.

<FIG> is a block diagram illustrating an exemplary connection load balancer. Referring to <FIG>, connection load balancer <NUM> includes at least one processor <NUM> and a memory <NUM>. The S1/N2 connection distributor <NUM> may be implemented in software stored in memory <NUM> to perform the S1/N2 connection distribution steps described herein. Connection load balancer <NUM> also includes signaling or Ethernet interfaces <NUM> and <NUM> that receive the incoming S1/N2 connection requests from RAN nodes. In one example, connection load balancer <NUM> associates MAC addresses of signaling interfaces <NUM> and <NUM> with published IP addresses that are also associated with loopback interfaces of MME/AMF instances <NUM>. Connection load balancer <NUM> may also include an S1/N2 association cache or database <NUM> that is populated at least in part by connection load balancer <NUM> with associations between IP addresses of MME/AMF instances <NUM> selected to handle connections and remote RAN peer IP addresses and ports. For example, S1/N2 connection distributor <NUM> may create, in S1/N2 association database or cache <NUM>, associations between IP addresses of MME/AMF instances <NUM> assigned to connections and IP addresses and ports of remote RAN nodes extracted from source IP address and port fields of connection management messages.

<FIG> is a flow chart illustrating an exemplary method that may be implemented by connection load balancer <NUM> in distributing S1/N2 connections to mobility management nodes. Referring to <FIG>, in step <NUM>, the method includes publishing, by the connection load balancer, Internet protocol (IP) addresses for receiving S1/N2 connection requests and ingress messages from RAN nodes. For example, S1/N2 connection distributor <NUM> of connection load balancer <NUM> may broadcast gratuitous ARP messages or ICMPv6 neighbor discovery messages that associate published IP addresses with MAC addresses of the signaling interfaces of the connection load balancer at the border gateway. Using the IP address IP-Node-A in <FIG> and <FIG> as an example, connection load balancer <NUM> sends a gratuitous ARP messages or ICMPv6 neighbor discovery messages to border gateway <NUM> that associates the IP address IP-Node-A with the MAC address MAC-A, which is the MAC address of the signaling interface SIG-A <NUM> of connection load balancer <NUM>. Each MME/AMF instance <NUM> configures its loopback interface with the IP address IP-Node-A as an alias address. The reason that the loopback interface is used for adding the published IP addresses as alias on MME/AMF instances is that the loopback interface doesn't have a MAC address, and an ARP reply or ICMPv6 neighbor discovery message response is not sent if ARP requests or ICMPv6 neighbor discovery requests are received for IP addresses configured as aliases on the loopback interface. Thus, multiple MME/AMF instances can configure same published IP addresses with no network conflicts. The reason that the IP addresses IP-Node-A and IP-Node-B are configured as alias addresses on the loopback interfaces of MME/AMF instances is to bind a transport layer with these addresses so that the MME/AMF instances will listen on this port for S1/N2 connection requests.

Returning to <FIG>, in step <NUM>, the method further includes maintaining, by the connection load balancer, connection loading measurements of the mobility management node instances for each group. In one example, connection load balancer <NUM> may maintain a group count for each MME/AMF instance <NUM> for all configured groups. The group count indicates the number of connections assigned to each MME/AMF instance <NUM> per group. Other loading measurements may be used without departing from the scope of the subject matter described herein.

In step <NUM>, the method further includes receiving, at the connection load balancer, a connection request message generated by a remote peer RAN node for initiating a connection with one of the mobility management node instances. For example, if the mobility management node instance is an MME, the connection request may be an S1 connection request. If the mobility management node instance is an AMF, the connection request may be an N2 connection request.

In step <NUM>, the method further includes applying, by the connection load balancer, a connection distribution algorithm to select a mobility management node instance to handle the connection request message and creating, by the connection load balancer, for protocol continuity, and in a database populated at least in part by the connection load balancer an association between the IP address of the selected mobility management node instance and an IP address and port of the remote RAN peer extracted from a source IP address and source port of the connection request message For example, the connection distribution algorithm may select the MME/AMF instance from MME/AMF instances with group count differences that exceed the connection distribution threshold to receive a new S1 or N2 connection request. Connection load balancer <NUM> may create in S1/N2 association cache or database <NUM> an association between the published IP address associated with the selected MME or AMF instance and an IP address and port of the remote RAN peer. Using <FIG> as an example, the association could between an IP address and port of RAN node <NUM> and the IP address IP-Node-A published on behalf of one of MME/AMF instances <NUM> selected to handle the connection.

In step <NUM>, the method further includes forwarding, by the connection load balancer, the connection request message to the mobility management node instance selected using the connection distribution algorithm. For example, connection load balancer <NUM> may forward the connection request to the MME/AMF instance selected in step <NUM> and update the group count for the MME/AMF instance. Connection load balancer create an association record in its cache for protocol continuity for the subsequent messages on this S1/N2 connection can be forwarded to this MME/AMF instance. In the message transmitted to the MME/AMF instance, the connection load balancer may include its MAC address as the source MAC address.

The connection load balancer described herein provides at least the following advantages:
The connection load balancer described herein can be used to distribute S1/N2 connections among MME/AMF instances, which reduces the impact of changes in MME/AMF configuration on sending nodes. That is, the sending nodes or remote peers are not required to re-establish connections with new IP addresses when MME/AMF instances go down or if new MME/AMF instances are established.

The connection load balancer can be used to solve this problem by using single interface to a remote peer and managing the internal interfaces irrespective of whether one IP interface card or multiple interface cards are used for setting up S1/N2 connections.

Locating the connection load balancer upstream from MME/AMF nodes in cloud/on-premise environments has the following benefits:.

Claim 1:
A method for distributing connections (<NUM>) to mobility management node instances (<NUM>), the method comprising:
publishing (<NUM>), by a connection load balancer (<NUM>), Internet protocol, IP, addresses for receiving connection requests and ingress messages from radio access network, RAN, nodes (<NUM>), wherein the IP addresses are alias IP addresses associated with loopback interfaces of the mobility management node instances and the mobility management node instances listen for messages addressed to the alias IP addresses;
maintaining (<NUM>), by the connection load balancer, connection loading measurements of the mobility management node instances;
receiving (<NUM>), at the connection load balancer, a connection request message generated by a RAN node for initiating a connection with one of the mobility management node instances, the connection request message being addressed to one of the alias IP addresses;
applying (<NUM>), by the connection load balancer, a connection distribution algorithm to select a mobility management node instance to handle the connection request message;
creating, by the connection load balancer, for protocol continuity, and in a cache or database (<NUM>) populated at least in part by the connection load balancer, an association between an IP address of the selected mobility management node instance and an IP address and port of the RAN node extracted from a source IP address and source port field of the connection request message; and
forwarding (<NUM>), by the connection load balancer, the connection request message to the mobility management node instance selected using the connection distribution algorithm.