Patent Description:
Third Generation Partnership Project (3GPP) Release <NUM> specifies the evolved packet system (EPS). EPS is based on the long-term evolution (LTE) radio network and the evolved packet core (EPC). It was originally intended to provide voice and mobile broadband (MBB) services but has continuously evolved to broaden its functionality. Since Release <NUM> narrowband Internet of Things (NB-IoT) and LTE for machine type communication (LTE-M) are part of the LTE specifications and provide connectivity to massive machine type communications (mMTC) services.

3GPP Release <NUM> developed the first release of the fifth generation (<NUM>) wireless network system. <NUM> is a new generation radio access technology intended to serve use cases such as enhanced mobile broadband (eMBB), ultra-reliable and low latency communication (URLLC) and mMTC. <NUM> includes the new radio (NR) access stratum interface and the <NUM> core network (5GC). The NR physical and higher layers reuse parts of the LTE specification, and to that add needed components when motivated by the new use cases.

Mobile networks traditionally serve devices on the ground, but interest and business case for using mobile networks, including both existing LTE networks and emerging <NUM> networks, to provide connectivity to low altitude drones have been growing fast. With their mobility, agility, and flexibility, drones are widely used in various applications. In particular, drone user equipments (UEs) play a key role in a number of scenarios such as package delivery, remote sensing, and surveillance applications. To operate properly, flying drones need to be effectively supported via cellular networks (referred to as cellular-connected drones) to ensure seamless connectivity and low-latency communications. In this regard, there is a need for efficient handover (HO) mechanisms for drone mobility management to provide reliable communications between base stations (BSs) and drones.

Handover is a process in which user-base station association in a cellular network changes so that the user maintains their connectivity while moving through different cells. A UE is either in an idle/inactive or connected mode. In idle mode, the UE camps on a cell and does not have any active signaling or data-bearers to the network. However, when in connected mode the network allocates resources to the UEs and handover involves active signaling on the data and control channels between users and BSs and signaling between source and target cells, thus causing overhead in the cellular network.

In principle, the number of handovers depends on the various factors such as the number of BSs, location, speed, and trajectory of mobile users, reference signal received power (RSRP) variations, reference signal received quality (RSRQ) variations, and the handover mechanism. While evaluating the mobility performance and handover mechanisms, several key metrics besides the number of handovers can be considered, such as: signal quality, signaling overhead, the number of radio link failures (RLF), and the rate of ping-pong handovers.

The following handover procedure in LTE is used as an example to illustrate how connected mode mobility typically works. The network triggers the handover procedure, e.g. based on radio conditions and load. To facilitate this, the network may configure the UE to perform measurement reporting. The network may also initiate handover blindly, i.e. without having received measurement reports from the UE.

Before sending the handover message to the UE, the source eNB prepares one or more target cells. The source eNB selects the target PCell.

The target eNB generates the message used to perform the handover, i.e. the message including the AS-configuration to be used in the target cell(s). The source eNB transparently forwards the handover message/information received from the target to the UE. After receiving the handover message, the UE attempts to access the target PCell at the first available random access channel (RACH) occasion, i.e. the handover is asynchronous, or at the first available physical uplink shared channel (PUSCH) occasion if rach-Skip is configured.

Upon successful completion of the handover, the UE sends a message used to confirm the handover. Upon having detected handover failure, the UE attempts to resume the radio resource control (RRC) connection either in the source PCell or in another cell using the RRC re-establishment procedure.

Compared to ground users, mobility support for aerial users (e.g., drones) is more challenging because of the following reasons. Unlike ground users, drones can move in any direction in three-dimensions (3D), can have arbitrary trajectory, and typically move faster than ground users.

Additionally, BSs are mainly designed to serve ground users and thus their antennas are down-tilted. The main lobe of a BS antenna thus covers a large part of the surface area of the cell to improve performance for terrestrial UEs. Accordingly, at ground level the strongest site is typically the closest one. A drone UE on the other hand may be frequently served by the sidelobes of BS antennas, which have lower antenna gains. The coverage areas of the sidelobes may be small and the signals at the edges may drop sharply due to deep antenna nulls. At a given location, the strongest signal might come from a faraway BS, if the gain of the sidelobes of the closer BSs to the drone UE is much weaker. Additionally, the side lobes of BSs may not fully cover the sky and there can be coverage holes (space without coverage service) in the sky that can cause failure of drone's connectivity.

An illustration of some of the above effects can be seen in <FIG>, which illustrates the maximum-received-power-based cell association patterns at the ground level and at the heights of <NUM>, <NUM>, and <NUM>. At the higher heights, the coverage areas become fragmented and the fragmentation pattern is determined by the lobe structures of the BS antennas. Such fragmented coverage area provided by different BSs makes mobility support in the sky more difficult and can result in frequent handovers.

Another issue in the sky is interference. As the height increases, more BSs have line of sight propagation conditions to drone UEs. As a result, the drone UEs may generate more uplink interference to the neighbor cells while experiencing more downlink interference from the neighbor cells. Due to the increased interference, signal-to-interference-plus-noise ratio (SINR) may become quite poor at certain heights. The degraded SINR might lead to more RLFs. It might also result in more handover failures because measurement reports, handover commands, etc., may get lost during the handover execution procedure.

Given the importance of cellular-connected drones, the 3GPP Release <NUM>, TR <NUM> studied the potential support of LTE for providing connectivity for drones. One of the main goals of the study was to analyze drone mobility performance and to identify efficient handover signaling/mechanisms. The result of this study shows that mobility support for drones is one of the challenging aspects in LTE-to-drone communications.

There currently exist certain challenges. For example, the existing mobility management procedure has difficulties in providing robust mobility support for ubiquitous 3D coverage, particularly for providing connectivity to low altitude drones. To illustrate the challenges of mobility support for drone UEs, <FIG> illustrates a simulated example mobility trace for a drone UE moving away from the coverage of a BS antenna sidelobe at the speed of <NUM>/h and at the height of <NUM>.

The upper subfigure of <FIG> illustrates the RSRP measurements by the drone UE, and the bottom subfigure shows the time varying trace of the serving cell SINR. Each numbered RSRP trace (e.g., <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) corresponds to a different cell. In <FIG>, the vertical dashed line at the beginning (t=<NUM>) marks cell selection of cell <NUM>. At about t = <NUM>, the serving cell RSRP begins to drop, and it drops by <NUM> dB within <NUM>. After <NUM>, the neighboring cells become stronger than the serving cell. However, to trigger handover measurement reports, some neighbor cell RSRP should be X dB better than the serving cell, where X is set to <NUM> in the simulation. From the RSRP traces, it is evident that the RSRPs of the neighbor cells stay relatively low, and none of them is at least <NUM> dB better than the serving cell before the drone UE declares RLF at t = <NUM> (marked by the vertical dashed line) due to poor serving cell SINR.

Furthermore, the embodiments of the invention are those defined by the claims.

Based on the description above, certain challenges currently exist with three-dimensional (3D) mobility. Certain aspects of the present disclosure and their embodiments may provide solutions to these or other challenges. For example, particular embodiments include a robust handover (HO) mechanism for 3D mobility management (e.g., for drone user equipments (UEs)) based on reinforcement learning while capturing the inherent tradeoff between drone connectivity performance and handover signaling overhead.

In particular embodiments, the handover mechanism uses various information, such as real-time reference signal received power (RSRP)/reference signal received quality (RSRQ), prior RSRP/RSRQ data, UE's 3D trajectory, and UE's speed to provide effective handover rules for seamless 3D connectivity with small handover signaling overhead. Moreover, particular embodiments jointly optimize handover decisions and UE trajectory in applications in which UE routes are not pre-defined or fixed.

In general, particular embodiments use a machine-learning assisted mobility management procedure for efficient 3D mobility in cellular systems. Particular embodiments include methods for each stage of the procedure including data collection, model training, model usage and model update with and without UE assistance.

According to some embodiments, a method performed by a network node for mobility management comprises obtaining data samples for modeling a wireless network environment that comprises a plurality of cells and building a machine learning model of the wireless network using the obtained data samples. The machine learning model (e.g., reinforcement learning model) is trained to determine a sequence of handovers for a wireless device among the plurality of cells for the wireless device to traverse from a source cell to a destination cell. The method further comprises receiving mobility information for a wireless device, determining one or more handover operations for the wireless device based on the mobility information, and transmitting the one or more handover operations to the wireless device.

<CIT> discloses a switching algorithm based on machine learning in a UDN, in which a pre-switching request is sent to a predicted target micro cell.

<CIT> discloses a network node operable to generate, by adaptive programming, an algorithm for adjusting deployed base station operating characteristics to assist in providing desired base station operating characteristics.

<CIT> discloses a mobile management handover system includes circuitry that determines available resources of candidate target handover cells of a user device.

<CIT> discloses a communication device configured to receive an uplink radio transmission in a first waveform format and transmit the uplink radio transmission to a network access node with a preamble in a second waveform format to protect the uplink radio transmission from collisions.

A non-patent literature, <NPL> et al; discloses a handover scheme that utilizes a predictive model based on a Multilayer Feed-forward Neural Network (MFNN) with backpropagation algorithm.

In particular embodiments, obtaining data samples for modeling the wireless network environment comprises obtaining one or more of base station locations and base station antenna patterns. Obtaining data samples may comprise obtaining propagation environment information including a location of obstacles in the propagation environment. Obtaining data samples may comprise obtaining wireless signal characteristics for a plurality of locations in the wireless network environment. The wireless signal characteristics may include one or more of reference RSRP, RSRQ, and signal to interference plus noise ratio (SINR). Obtaining data samples may comprise obtaining data samples from one or more wireless devices.

In particular embodiments, the machine learning model is trained to determine a sequence of handovers for a wireless device among the plurality of cells for the wireless device to traverse from a source cell to a destination cell by minimizing one or more of a number of handovers, radio link failure (RLF), and ping-pong handovers.

In particular embodiments, the mobility information comprises one or more of a position of the wireless device, a velocity of the wireless device, a mobility pattern of the wireless device, a wireless signal quality of a serving cell for the wireless device, and a wireless signal quality of a neighbor cell for the wireless device.

In particular embodiments, the mobility information may comprise a destination, and determining one or more handover operations for the wireless device comprises determining a sequence of handover operations for the wireless device to navigate to the destination. The network node may transmit the sequence of handover operations to the wireless device in advance of the wireless device navigating to the destination. The network node may transmit the sequence of handover operations to the wireless device as the wireless device navigates to the destination.

In particular embodiments, the network node comprises one of a base station and a core network node.

According to some embodiments, a network is capable of mobility management. The network node comprises processing circuitry operable to perform any of the network node methods described above.

Also disclosed is a computer program product comprising a non-transitory computer readable medium storing computer readable program code, the computer readable program code operable, when executed by processing circuitry to perform any of the methods performed by the network node described above.

According to some embodiments, a method performed by a wireless device for mobility management comprises transmitting mobility information to a network node. The network node comprises a machine learning model trained to determine a sequence of handovers for a wireless device among a plurality of cells for the wireless device to traverse from a source cell to a destination cell. The method further comprises receiving one or more handover operations from the network node machine learning model and performing a handover according to one of the received one or more handover operations.

In particular embodiments, the machine learning model is trained to determine a sequence of handovers for a wireless device among the plurality of cells for the wireless device to traverse from a source cell to a destination cell by minimizing one or more of a number of handovers, RLF, signaling overhead, and ping-pong handovers.

In particular embodiments, the mobility information comprises one or more of a position of the wireless device, a velocity of the wireless device, a wireless signal quality of a serving cell for the wireless device, and a wireless signal quality of a neighbor cell for the wireless device.

In particular embodiments, the mobility information comprises a destination, and the received one or more handover operations comprise a sequence of handover operations for the wireless device to navigate to the destination. The wireless device may receive the sequence of handover operations in advance of navigating to the destination. The wireless device may receive the sequence of handover operations while navigating to the destination.

According to some embodiments, a wireless device is capable of mobility management. The wireless device comprises processing circuitry operable to perform any of the wireless device methods described above.

Also disclosed is a computer program product comprising a non-transitory computer readable medium storing computer readable program code, the computer readable program code operable, when executed by processing circuitry to perform any of the methods performed by the wireless device described above.

Certain embodiments may provide one or more of the following technical advantages. For example, particular embodiments for supporting 3D mobility (e.g., for drone UEs) facilitate efficient and flexible handover decisions while considering UE connectivity as well as handover signaling.

In fixed-drone trajectory scenarios, the network uses machine learning tools to specify when handover must be done to maintain connectivity while reducing handover signaling and ping-pong handover rate. In addition, when possible, the UE trajectory (e.g., drone's route) can be optimized along with handover rules to improve the drone mobility support.

In summary, particular embodiments include at least the following advantages: (a) improving UE connectivity in 3D by using various information such as RSRP/RSRQ values, UE route, UE speed, flight/mobility regulations (e.g., no-fly zones), and information about the environment (e.g., location and size of buildings); (b) reducing the number of handovers and improving the robustness of handovers; (c) joint design of UE 3D trajectory (e.g., in drone scenarios) and handover rules to enhance UE mobility performance in term of various metrics such as RSRP/RSRQ values, signaling overhead, the number of radio link failures, and the rate of ping-pong handovers; and (d) using a feedback mechanism (e.g., event-triggered feedback) to dynamically update handover decisions/parameters to improve the performance.

Particular embodiments support UE mobility in 3D, including drone UEs or terrestrial UEs moving in 3D (e.g., uphill).

As described above, certain challenges currently exist with three-dimensional (3D) mobility. For example, unlike ground users, drones can move in any direction in three-dimensions, can have arbitrary trajectory, and typically move faster than ground users. Additionally, base stations (BSs) are mainly designed to serve ground users and thus their antennas are down-tilted. The main lobe of a base station antenna thus covers a large part of the surface area of the cell to improve performance for terrestrial user equipment (UEs).

A drone UE, on the other hand, may be frequently served by the sidelobes of base station antennas, which have lower antenna gains. The coverage areas of the sidelobes may be small and the signals at the edges may drop sharply due to deep antenna nulls. At a given location, the strongest signal might come from a faraway base station. Additionally, the side lobes of base stations may not fully cover the sky resulting in coverage holes that can cause drone connectivity failure.

Certain aspects of the present disclosure and their embodiments may provide solutions to these or other challenges. For example, particular embodiments use a machine-learning assisted mobility management procedure for efficient 3D mobility in cellular systems. Particular embodiments include methods for each stage of the procedure including data collection, model training, model usage and model update with and without UE assistance.

Particular embodiments are described more fully with reference to the accompanying drawings.

A common example of a 3D mobility scenario is a cellular-connected drone system in which drone UEs are served by ground base stations. An example is illustrated in <FIG>.

<FIG> is a network diagram illustrating 3D mobility support in a cellular-connected drone scenario. Drone <NUM> moves from a start location to a destination on a pre-defined fixed route or a flexible route (which is not fixed). During the movement, drone <NUM> connects with base stations <NUM> to exchange necessary information for control, navigation, and communication. To provide seamless connectivity to mobile drone <NUM>, the drone's connection may change from one base station <NUM> to another during a handover process.

Particular embodiments described herein include an efficient mobility management mechanism that can ensure connectivity requirements while reducing costs (e.g., overhead, radio link failures (RLF), ping-pong handover, etc.) associated with the handover process.

Particular embodiments include a method based on reinforcement learning to achieve efficient 3D mobility. The method involves data collection for developing a machine learning model, building and training the model, deploying the model, and updating the model. A flow chart of the proposed method is illustrated in <FIG>.

<FIG> is a flowchart illustrating an example reinforcement learning based 3D mobility management method. A first step includes data collection for developing the machine learning model. To build the machine learning model, data is collected. The types of data may include reference signal receive power (RSRP), reference signal receive quality (RSRQ), and signal to interference plus noise ratio (SINR) values within a desired 3D space, network layout and configuration such as base station locations, base station antennas patterns, and propagation environment such as the 3D locations of buildings/obstacles (including height and shape of the buildings).

In some embodiments, existing data can be used. New data can be collected and integrated with existing data to enhance the quality and accuracy of the data. Particular embodiments focus on how new RSRP/RSRQ/SINR data can be collected.

In some embodiments, the gNB configures one or more UEs to measure and report RSRP/RSRQ/SINR and the UE's 3D position periodically or in an event triggered manner.

In some embodiments, the gNB configures one or more UEs to measure and report RSRP/RSRQ/SINR periodically or in an event triggered manner, and the gNB estimates the UE's 3D position corresponding to the reported measurements.

In some embodiments, the network deploys one or more UEs dedicated to data collection. These UEs measure RSRP/RSRQ/SINR and their 3D positions periodically. The UEs may store the data locally which are fetched after the data collection mission. Alternatively, the UEs transmit the data to the network during the data collection mission.

A second step includes building and training the model. In general, always connecting to the strongest base station (i.e., the base station that provides maximum RSRP, RSRQ or SINR) may not be efficient for UE connectivity and handover signaling overhead. Making a handover decision solely based on the current maximum RSRP/RSRQ/SINR can lead to many subsequent handovers, which is not efficient. In addition, it can cause ping-pong handovers and connectivity failure. Considering this scenario, in particular embodiments, an efficient handover mechanism uses an optimal sequential decision-making scheme to build connections between the UE and the potential serving base stations.

To achieve optimal sequential handover decision-making for supporting 3D mobility (e.g., for drone UEs), a reinforcement learning (RL) model is built. In RL, an agent interacts with an environment by choosing actions based on the environment's current state (or an observation of the state). For a performed action in a state, the agent receives feedback in terms of a scalar reward and the environment transits to a new state. The reward and new state are stochastically determined by the dynamics of the environment, which in general are not known to the agent. The goal for the agent is to find an optimal policy to maximize the total reward.

In some embodiments, the mobility management RL model uses UE's position, UE's velocity, the UE's serving cell RSRP/RSRQ/SINR values, and the UE's neighbor cells' RSRP/RSRQ/SINR as the state of the model.

In some embodiments, the mobility management RL model uses a scalar reward function which depends on which serving cell for the UE to connect to, the corresponding RSRP/RSRQ/SINR values of the potential serving cell, handover signaling, and the loads of the cells.

In some embodiments, the mobility management RL model uses a reward function to account for real-time RSRP/RSRQ/SINR values, number of handovers, RLF, and ping-pong handovers.

In some embodiments, the mobility management RL model uses a set of candidate cells that the UE may connect to as the action space.

In some embodiments, the mobility management RL model uses a set of cells that the UE may connect to and a set of UE's movement actions (moving directions, acceleration/deceleration) as the action space.

After formulating the mobility management RL model, step <NUM> is to train the model using the data so that the model can approximate the value functions. For example, the optimal action value function denoted as Q*(s, a), which is the expected return when taking action a in state s and following an optimal policy π*. Once Q*(s, a) values are computed, the agent can act optimally as π*(s) = argmaxa Q*(s, a). Possible actions include performing handover and UE's movement, among others.

In some embodiments, the optimal action values are computed by Q-learning algorithm. In some embodiments, a neural network is trained as a function approximator for either the optimal action values or the optimal policy of the agent. This is a deep reinforcement learning approach. Training the neural network involves data preprocessing and normalization, choosing learning rate, ε-greedy parameter (balance exploitation and exploration), batch size, memory size, optimizer, and activation function for the neural network. An example is illustrated in <FIG>.

<FIG> illustrates an example neural network for approximating the optimal RL action values. In some embodiments, deep Q-learning network (DQN) is used to determine Q values of all possible actions and then identify the optimal action.

In some embodiments, a central entity stores all the data and trains a single machine learning model. Such an entity could be a network node, such as a base station or a core network node, or a device in the cloud.

In some embodiments, the machine learning models can be trained per cell or group of cells. In this case, each cell or group of cells can run a model locally and exchange the outputs with neighbors to collaboratively carry out the 3D mobility management.

In general, the trained RL model will output sequential handover decisions based on inputs, as illustrated in <FIG>.

<FIG> is a block diagram illustrating input and output of the reinforcement learning model. The inputs may include UE position and/or velocity, UE serving cell RSRP/RSRQ/SINR, and UE neighbor cell RSRP/RSRQ/SINR. The outputs may include sequential handover decisions for the UE.

Returning to <FIG>, a fourth step is to deploy the model and a fifth step is to use the model. For example, after the model is built, the next step is to deploy and use the model in networks.

In some embodiments, the model is deployed and executed at a central entity, which could be a network node or a device in the cloud, i.e., the central entity is the RL agent. In some embodiments, the model is deployed and executed at each gNB, i.e., the gNB is the RL agent. In some embodiments, the model is signaled to the UE that executes the model, i.e., the UE is the RL agent.

If information of all states such as UE trajectory is known a priori, the mobility management decisions can be determined using the model in advance. The network can communicate the decisions to the UE if the model is not executed at the UE. In general, the mobility management decisions are determined on the fly.

In some embodiments, the agent determines the decisions periodically. In some embodiments, the RL agent determines the decisions in an event triggered manner. For example, the agent determines a new action if the UE has moved more than a threshold distance from the previous location.

In some embodiments, the mobility management decisions are fully controlled by the network. In this case, the RL agent passes the output of the model to the network that determines the final mobility management decision.

Some embodiments use the model with UE assistance. A legacy mobility management procedure in LTE/NR is typically dependent on UE measurement reports, which are compiled based on signal strength measurements for serving and neighbor cells performed by the UE. While the accuracy of mobility management decisions can be improved with proper training/testing of the model, an implausible decision from the model cannot be ruled out. This calls for UE assistance to weed out poor mobility management decisions predicted by the model.

In one embodiment, the UE executes the mobility management decision conveyed by the network or determined by itself if the UE is the RL agent, when one or more predefined/preconfigured conditions are satisfied. These conditions could be: (a) the target cell signal strength/quality satisfies predefined condition(s) (e.g., the strength/quality exceeds certain preconfigured threshold(s)); (b) the source cell signal strength/quality satisfy predefined condition(s) (e.g., the strength/quality is lower than certain preconfigured threshold(s)); and (c) the source cell signal strength/quality are better than the target cell signal strength/quality by some thresholds.

In some embodiments, the UE indicates to the network whether it agrees with the mobility management decision output by the model. For example, when the conditions are satisfied, the UE agrees with the decision (ACK), otherwise it does not (NACK). The UE may feedback <NUM>-bit information, for example, to indicate if it agrees with the handover decision. UE may feedback additional information related to the conditions which are not satisfied, or other UE-specific information such as UE altitude or height which may help the network to improve the model.

In some embodiments, the network leverages the UE feedback to improve the model. For example, based on the UE explicit feedback the network decides whether to use the model or legacy mobility management procedure.

In some embodiments, the network decides whether to use the model or legacy mobility management procedure based on implicit feedback such as RLF statistics, ping-pong handovers, or a combination of the feedback collected from various mechanisms.

The network may implement this decision (model or legacy mobility management procedure) at various levels such as UE-specific, specific to a group of UEs (e.g., drone UEs above or below certain altitudes, etc.), and cell level.

Another step includes updating the model. While the model is deployed and used in the network, the model can be updated by adapting to any changes in the environment/scenario and incorporating new data. The model update can be performed periodically or in an event-triggered manner. For example, the event may be that the number of UE feedbacks that disagree with the model output exceeds a threshold. This may indicate the model is not performing well and thus triggers an update. During the update, the network may switch the UE back to legacy mobility management mode.

In one embodiment, the network updates the mobility management model periodically or in an event-triggered manner.

<FIG> illustrates an example wireless network, according to certain embodiments.

These components work together to provide network node and/or wireless device functionality, such as providing wireless connections in a wireless network.

Interface <NUM> is used in the wired or wireless communication of signaling and/or data between network node <NUM>, network <NUM>, and/or WDs <NUM>.

As used herein, wireless device (WD) refers to a device capable, configured, arranged and/or operable to communicate wirelessly with network nodes and/or other wireless devices. Unless otherwise noted, the term WD may be used interchangeably herein with user equipment (UE). Communicating wirelessly may involve transmitting and/or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and/or other types of signals suitable for conveying information through air.

In some embodiments, a WD may be configured to transmit and/or receive information without direct human interaction. For instance, a WD may be designed to transmit information to a network on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the network.

Examples of a WD include, but are not limited to, a smart phone, a mobile phone, a cell phone, a voice over IP (VoIP) phone, a wireless local loop phone, a desktop computer, a personal digital assistant (PDA), a wireless cameras, a gaming console or device, a music storage device, a playback appliance, a wearable terminal device, a wireless endpoint, a mobile station, a tablet, a laptop, a laptop-embedded equipment (LEE), a laptop-mounted equipment (LME), a smart device, a wireless customer-premise equipment (CPE). a vehicle-mounted wireless terminal device, etc. A WD may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), vehicle-to-everything (V2X) and may in this case be referred to as a D2D communication device.

As yet another specific example, in an Internet of Things (IoT) scenario, a WD may represent a machine or other device that performs monitoring and/or measurements and transmits the results of such monitoring and/or measurements to another WD and/or a network node. The WD may in this case be a machine-to-machine (M2M) device, which may in a 3GPP context be referred to as an MTC device. As one example, the WD may be a UE implementing the 3GPP narrow band internet of things (NB-IoT) standard. Examples of such machines or devices are sensors, metering devices such as power meters, industrial machinery, or home or personal appliances (e.g. refrigerators, televisions, etc.) personal wearables (e.g., watches, fitness trackers, etc.).

In other scenarios, a WD may represent a vehicle or other equipment that is capable of monitoring and/or reporting on its operational status or other functions associated with its operation. A WD as described above may represent the endpoint of a wireless connection, in which case the device may be referred to as a wireless terminal. Furthermore, a WD as described above may be mobile, in which case it may also be referred to as a mobile device or a mobile terminal.

Radio front end circuitry <NUM> is connected to antenna <NUM> and processing circuitry <NUM> and is configured to condition signals communicated between antenna <NUM> and processing circuitry <NUM>.

The benefits provided by such functionality are not limited to processing circuitry <NUM> alone or to other components of WD <NUM>, but are enjoyed by WD <NUM>, and/or by end users and the wireless network generally.

In some embodiments, processing circuitry <NUM> and device readable medium <NUM> may be integrated.

User interface equipment <NUM> is configured to allow input of information into WD <NUM> and is connected to processing circuitry <NUM> to allow processing circuitry <NUM> to process the input information. Using one or more input and output interfaces, devices, and circuits, of user interface equipment <NUM>, WD <NUM> may communicate with end users and/or the wireless network and allow them to benefit from the functionality described herein.

Although the subject matter described herein may be implemented in any appropriate type of system using any suitable components, the embodiments disclosed herein are described in relation to a wireless network, such as the example wireless network illustrated in <FIG>. For simplicity, the wireless network of <FIG> only depicts network <NUM>, network nodes <NUM> and 160b, and WDs <NUM>, 110b, and 110c. In practice, a wireless network may further include any additional elements suitable to support communication between wireless devices or between a wireless device and another communication device, such as a landline telephone, a service provider, or any other network node or end device. Of the illustrated components, network node <NUM> and wireless device (WD) <NUM> are depicted with additional detail. The wireless network may provide communication and other types of services to one or more wireless devices to facilitate the wireless devices' access to and/or use of the services provided by, or via, the wireless network.

<FIG> illustrates an example user equipment, according to certain embodiments.

Certain UEs may use all the components shown in <FIG>, or only a subset of the components.

<FIG> is a flowchart illustrating an example method in a network node, according to certain embodiments. In particular embodiments, one or more steps of <FIG> may be performed by network node <NUM> described with respect to <FIG>. In some embodiments, one or more steps of <FIG> may be performed by a core network node.

In general, the network node will collect data samples regarding antenna coverage for the network and use the data samples to build a machine learning model that can be used to predict efficient handover locations and predict routes through the wireless network to avoid coverage holes.

The method begins at step <NUM>, where the network node (e.g., network node <NUM>) obtains data samples for modeling a wireless network environment that comprises a plurality of cells. For example, the network node may obtain one or more of base station locations and base station antenna patterns. Obtaining data samples may comprise obtaining propagation environment information including a location of obstacles (e.g., buildings, etc.) in the propagation environment. Obtaining data samples may comprise obtaining wireless signal characteristics for a plurality of locations in the wireless network environment. The wireless signal characteristics may include one or more of RSRP, RSRQ, and SINR. Obtaining data samples may comprise obtaining data samples from one or more wireless devices.

For example, wireless devices may be deployed in advance to collect information and send data to the network node. In another example, the wireless devices may report data in the course of their normal movement.

In some embodiments, the network node obtains data samples according to any of the embodiments and examples described above, such as those described with respect to <FIG>.

At step <NUM>, the network node builds a machine learning model of the wireless network using the obtained data samples. The machine learning model is trained to determine a sequence of handovers for a wireless device among the plurality of cells for the wireless device to traverse from a source cell to a destination cell.

In particular embodiments, the machine learning model (e.g., reinforcement learning model) is trained to determine a sequence of handovers for a wireless device among the plurality of cells for the wireless device to traverse from a source cell to a destination cell by minimizing one or more of a number of handovers, radio link failure (RLF), signaling overhead, and ping-pong handovers, according to any of the embodiments and examples described above, such as those described with respect to <FIG>.

At this point the machine learning model is trained and is ready to be deployed for use by wireless devices (e.g., drones) in the network. The network node can receive mobility information from the wireless devices, determine route and handover information, and provide the response to the wireless devices.

At step <NUM>, the network node receives mobility information for a wireless device. In particular embodiments, the mobility information comprises one or more of a position of the wireless device, a velocity of the wireless device, a mobility pattern of the wireless device, a destination for the wireless device, a wireless signal quality of a serving cell for the wireless device, and a wireless signal quality of a neighbor cell for the wireless device. The mobility information may include any of the mobility information described with respect to any of the embodiments and examples described above, such as those described with respect to <FIG>.

At step <NUM>, the network node determines one or more handover operations for the wireless device based on the mobility information. For example, based on the mobility information, the learning model is able to determine an optimal next hop and/or an optimal route to a destination. As one example, the mobility information may comprise a destination, and determining one or more handover operations for the wireless device comprises determining a sequence of handover operations for the wireless device to navigate to the destination.

The network node may determine one or more handover operations for the wireless device according to any of the embodiments and examples described above, such as those described with respect to <FIG>.

At step <NUM>, the network node transmits the one or more handover operations to the wireless device. For example, the network node may transmit a next hop handover or a sequence of handovers. The network node may transmit the sequence of handover operations to the wireless device in advance of the wireless device navigating to the destination. The network node may transmit the sequence of handover operations to the wireless device as the wireless device navigates to the destination. The network node may update an original sequence of handover operations as the wireless device navigates to the destination.

Modifications, additions, or omissions may be made to method <NUM> of <FIG>. Additionally, one or more steps in the method of <FIG> may be performed in parallel or in any suitable order.

<FIG> is a flowchart illustrating another example method in a wireless device, according to certain embodiments. In particular embodiments, one or more steps of <FIG> may be performed by wireless device <NUM> described with respect to <FIG>.

The method begins at step <NUM>, where the wireless device (e.g., wireless device <NUM>) transmitting mobility information to a network node. For example, the wireless device may transmit mobility information to the network node as described with respect to step <NUM> of <FIG>. For example, the network node may include a machine learning model trained to determine a sequence of handovers for a wireless device among a plurality of cells for the wireless device to traverse from a source cell to a destination cell as described with respect to <FIG> and <FIG>. The network node may determining one or more handover operations for the wireless device based on the mobility information.

At step <NUM>, the wireless device receives one or more handover operations from the network node machine learning model. For example, the wireless device may receive one or more handover operations as described with respect to step <NUM> of <FIG>.

At step <NUM>, the wireless device performs a handover according to one of the received one or more handover operations. For example, the wireless device may perform a handover to a next hop on a route to its destination.

<FIG> illustrates a schematic block diagram of two apparatuses in a wireless network (for example, the wireless network illustrated in <FIG>). The apparatuses include a wireless device and a network node (e.g., wireless device <NUM> and network node <NUM> illustrated in <FIG>). Apparatuses <NUM> and <NUM> are operable to carry out the example methods described with reference to <FIG> and <FIG>, respectively, and possibly any other processes or methods disclosed herein. It is also to be understood that the methods of <FIG> and <FIG> are not necessarily carried out solely by apparatuses <NUM> and/or <NUM>. At least some operations of the methods can be performed by one or more other entities.

Virtual apparatuses <NUM> and <NUM> may comprise processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and/or data communications protocols as well as instructions for carrying out one or more of the techniques described herein, in several embodiments.

In some implementations, the processing circuitry may be used to cause receiving module <NUM>, determining module <NUM>, transmitting module <NUM>, and any other suitable units of apparatus <NUM> to perform corresponding functions according one or more embodiments of the present disclosure. Similarly, the processing circuitry described above may be used to cause receiving module <NUM>, determining module <NUM>, transmitting module <NUM>, and any other suitable units of apparatus <NUM> to perform corresponding functions according one or more embodiments of the present disclosure.

As illustrated in <FIG>, apparatus <NUM> includes receiving module <NUM> configured to receive receiving one or more handover operations from a network node machine learning model, according to any of the embodiments and examples described herein. Determining module <NUM> is configured to a perform a handover according to one of the received one or more handover operations, according to any of the embodiments and examples described herein. Transmitting module <NUM> is configured to transmit mobility information to a network node, according to any of the embodiments and examples described herein.

As illustrated in <FIG>, apparatus <NUM> includes receiving module <NUM> configured to obtain data samples for modeling a wireless network environment that comprises a plurality of cells and to receive mobility information for a wireless device, according to any of the embodiments and examples described herein. Determining module <NUM> is configured to determine a machine learning model of the wireless network using the obtained data samples and training the machine learning model to determine a sequence of handovers for a wireless device among the plurality of cells for the wireless device to traverse from a source cell to a destination cell, according to any of the embodiments and examples described herein. Transmitting module <NUM> is configured to transmit one or more handover operations to a wireless device, according to any of the embodiments and examples described herein.

NFV may be used to consolidate many network equipment types onto industry standard high-volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

Host computer <NUM> may be under the ownership or control of a service provider or may be operated by the service provider or on behalf of the service provider.

<FIG> illustrates an example host computer communicating via a base station with a user equipment over a partially wireless connection, according to certain embodiments. Example implementations, in accordance with an embodiment of the UE, base station and host computer discussed in the preceding paragraphs will now be described with reference to <FIG>.

Connection <NUM> may be direct, or it may pass through a core network (not shown in <FIG>) of the telecommunication system and/or through one or more intermediate networks outside the telecommunication system.

While OTT connection <NUM> is active, the network infrastructure may further take decisions by which it dynamically changes the routing (e.g., based on load balancing consideration or reconfiguration of the network).

Wireless connection <NUM> between UE <NUM> and base station <NUM> is in accordance with the teachings of the embodiments described throughout this disclosure. One or more of the various embodiments improve the performance of OTT services provided to UE <NUM> using OTT connection <NUM>, in which wireless connection <NUM> forms the last segment. More precisely, the teachings of these embodiments may improve the signaling overhead and reduce latency, which may provide faster internet access for users.

A measurement procedure may be provided for monitoring data rate, latency and other factors on which the one or more embodiments improve. In embodiments, sensors (not shown) may be deployed in or in association with communication devices through which OTT connection <NUM> passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above or supplying values of other physical quantities from which software <NUM>, <NUM> may compute or estimate the monitored quantities.

Additionally, or alternatively, in step <NUM>, the UE provides user data.

Modifications, additions, or omissions may be made to the systems and apparatuses disclosed herein without departing from the scope of the invention. The components of the systems and apparatuses may be integrated or separated. Moreover, the operations of the systems and apparatuses may be performed by more, fewer, or other components. Additionally, operations of the systems and apparatuses may be performed using any suitable logic comprising software, hardware, and/or other logic. As used in this document, "each" refers to each member of a set or each member of a subset of a set.

Modifications, additions, or omissions may be made to the methods disclosed herein without departing from the scope of the invention. The methods may include more, fewer, or other steps. Additionally, steps may be performed in any suitable order.

The foregoing description sets forth numerous specific details. It is understood, however, that embodiments may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the understanding of this description. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.

Although this disclosure has been described in terms of certain embodiments, alterations and permutations of the embodiments will be apparent to those skilled in the art. Accordingly, the above description of the embodiments does not constrain this disclosure. Other changes, substitutions, and alterations are possible without departing from the scope of this disclosure, as defined by the claims below.

Claim 1:
A method performed by a network node for mobility management, the method comprising:
obtaining (<NUM>) data samples for modeling a wireless network environment that comprises a plurality of cells;
building (<NUM>) a machine learning model of the wireless network using the obtained data samples, wherein the machine learning model is trained to determine a sequence of handovers for a wireless device among the plurality of cells for the wireless device to traverse from a source cell to a destination cell, wherein the machine learning model comprises a reinforcement learning, RL, model, wherein using the RL model, an agent interacts with the wireless network environment by choosing actions based on current state of the wireless network environment, the actions including performing handover;
receiving (<NUM>) mobility information for a wireless device, wherein the mobility information is used as a state of the RL model;
determining (<NUM>), using the machine learning model, one or more handover operations for the wireless device based on the mobility information; and
transmitting (<NUM>) the one or more handover operations to the wireless device.