Patent Description:
One of the main promises of the next-generation <NUM> networks is to preserve consistent Quality of Experience (QoE) while decreasing end-to-end latency. Simultaneously, Mobile Network Operators (MNOs) need to lower their Capital and Operational Expenses (CapEx and OpEx). To achieve these goals, the telecommunication industry has adopted virtualization technology to modularize and decouple network functions from other network functions and from specialized proprietary hardware.

Concurrently, container-based cloud native technologies, as a more lightweight solution to virtualization, has developed with the goal to maintain at least the same level of performance and isolation as virtualization provides, if not better. To mitigate several performance and stability challenges exposed by such a transformation, several optimization solutions have been developed to both reactively and predicatively respond to different traffic demand situations leading big data-driven intelligent agents that can manage resource allocation, routing, and mobility of network functions.

However, there exist certain challenges. For example, the extra-granularity of cloud-native applications increases the uncertainty about initial resource allocation configuration of each microservice, which can also be named as resource footprint configuration. If kept uncharted, it imposes multitudinous scaling operations and dramatically decreases Meantime Between Failure (MTBF) of microservices. <FIG> illustrates example Service Level Agreement (SLA) violations and scaling operations when resource requirements are too small or too big. More specifically, <FIG> demonstrates SLA violations over time as a result of uncharted resource requirements of a set of microservices in a <NUM> Cloud-native Network Function (CNF). <FIG> also shows the corresponding scaling operations (both scale-ups and scale-downs) to heal the failures in the Network Function Cloudification (NFC) system. <FIG> shows a relatively high failure rate or a relatively low MTBF as a result of wrong or no minimum resource requirements.

As another example, another challenge with CNFs, like Virtual Network Functions (VNFs), is the extra flexibility of placing them in the network: where to deploy Network Functions (NFs) in the network such that the total cost of system deployment is minimized and also users agreed QoS level assured.

Moreover, a telecommunications network is a complex system to configure. Each network node has many configuration parameters whose optimal values can change depending on many things including its geographical location, traffic load, and traffic types. It is even more difficult to optimize configuration for a CNF where each CNF contains many separately configurable microservices. The sheer number of combinations of configuration parameters of the microservices in the system makes it difficult for a human to configure the system in an optimal way.

Perhaps with smaller systems, obtaining an optimal initial configuration may be unnecessary. With their granularity and faster boot-up times, cloud-native microservices are very fluid and scalable. Simple threshold-based auto-scalers like Kubernetes' Horizontal Pod Autoscaler (HPA) or Elasticdocker can guarantee fair elasticity for most applications. However, auto-scaling leads to other problems like maintaining agreed QoS constraints while preventing multitudinous scale-up and scale-down actions as traffic demands change. A thorough understanding each microservice's resource usage characteristics and the system's dynamics from the beginning has utmost importance for maintaining a consistent QoE.

The Kubernetes Vertical Pod Autoscaler (VPA) Recommender is an attempt to solve this. It can be used to gather metrics from Prometheus and make pod-level generic resource limit recommendations for watched pods based on each pod's historical resource usage data. However, this solution is too generic and does not consider the interaction between microservices within a pod, nor the traffic model supported by the network function.

There have also been a proposal to produce a polynomial model to relate resource utilization of a virtual machine (VM) to its throughput. This technique uses a measure of how sensitive the VM is to different resources and how it behaves under resource contention to decide where to place the VM with respect to other VMs in an optimal way. However, this solution does not consider minimum resource requirements or application-level metrics.

Static tools and testing are available to minimize the guessing involved in how configuration parameters interact within a network node and how their values are affected by the presence or absence and performance of different microservices or other network nodes. However, it takes a significant amount of time and resources. There is a need to avoid these costs by obtaining the optimal initial configuration for a network node quickly without extensive testing. The need is even greater with the additional complexity of deploying the multiple microservices of a CNF in the network.

<CIT> discloses a proactive method for dynamically modifying monitoring infrastructure for dynamic service environment to meet customer SLA requirements. The method teaches using a monitoring framework to generate predictive models based on compliance data about a monitored system (e.g., one or more customer applications bound to one or more microservices in the dynamic service environment), current performance of the monitored system, and one or more underlying hosting environments associated with the monitored system. The predictive models are used for modifying monitoring policy specifying collection of monitoring data. The monitoring framework uses machine learning for predictive determination of time periods (i.e., monitoring windows) for dynamic monitoring.

<CIT> discloses a network service chain analytics system that provides an alarm when network elements do not meet required performance characteristics.

In conclusion, some shortcomings of existing solutions include the fact that there is no automated solution/framework in the industry or academia that extracts the QoS behaviour model of microservices in the cloud to recommend an initial resource footprint configuration on a given network and keep correcting its errors afterwards. Similar solutions only work on a specific problem by providing a tailor-made approach. These solutions use a static set of metrics and do not consider user-level SLAs. The solutions also have limitations on the number of resource types such as, for example, only considering a static set of flavors in their evaluations (e.g. Amazon flavors). Additionally, existing solutions lack a solution to place and allocate resources to each microservice in a CNF. Finally, none of the previously proposed solutions provide a way to minimize the benchmarking need to create a machine learning model needed to recommend the initial resource allocation configuration.

To address the foregoing problems with existing solutions, disclosed is systems and methods for model-based resource recommendation for cloud applications. For example, certain embodiments include comprehensive, automated and model-based solutions for modeling cloud-native applications such as NFC systems QoS metrics behaviour corresponding to its microservices resource allocation configuration. Such models may be used to recommend a configurable, predictive resource capacity and replication size for all microservices on a given substrate production network for each user.

The invention is defined by appended independent claims <NUM> and <NUM>. Preferred features are defined in dependent claims <NUM>-<NUM>. A method for performance modeling of microservices includes deploying the microservices within a network such that the microservices are communicatively coupled to generate at least one service chain for providing at least one service. Based on a resource allocation configuration, an initial set of training data is determined for the microservices within the network. At least a portion of data is excluded from the initial set of training data to generate a subset of training data. The step of excluding the portion of data from the initial set of training data to generate the subset of training data comprises: isolating resource allocation to each of the plurality of microservices; selecting one of the plurality of microservices; assigning a maximum respective resource allocation configuration to each of a plurality of resources associated with the microservices; determining a saturation point for the one of the plurality of microservices; and excluding a saturation area associated with the saturation point from the initial set of training data when generating the subset of training data. A QoS behaviour model is generated based on the subset of the training data.

A network node for performance modeling of a plurality of microservices includes processing circuitry configured to deploy the plurality of microservices within a network. The plurality of microservices are communicatively coupled to generate at least one service chain for providing at least one service. Based on a resource allocation configuration, an initial set of training data is determined for the plurality of microservices within the network. At least a portion of data is excluded from the initial set of training data to generate a subset of training data wherein excluding the portion of data from the initial set of training data to generate the subset of training data comprises the steps to:
isolate resource allocation to each of the plurality of microservices; select one of the plurality of microservices; assign a maximum respective resource allocation configuration to each of a plurality of resources associated with the microservices; determine a saturation point for the one of the plurality of microservices; and exclude a saturation area associated with the saturation point from the initial set of training data when generating the subset of training data. A QoS behaviour model is generated based on the subset of the training data.

The present disclosure may provide one or more technical advantages. For example, certain embodiments may provide an automated and configurable approach for modeling any given cloud-native system's dynamics by modeling the correlation between any QoS metrics with each and all microservice's resource types configuration (CPU size, memory size, etc.) such that the extracted regression models can predict the system's behaviour in a scaled environment with given contention ratios (even environments that had not been tested beforehand, but with the same or very similar hardware/OS/network configuration). This may dramatically reduce both OPEX and CAPEX such as by reducing the cost of dimensioning procedure, reducing SLA violations, reducing energy consumption and infrastructure cost, and increasing customer satisfaction as this gives a better estimate on infrastructure cost during initial SLA negotiation.

Another advantage may be that certain embodiments extract a behavioural model for microservices with any kind of QoS/performance behaviour, either linearly or non-linearly, or any other kind of distribution. Moreover, certain embodiments may provide a parametrized solution, so that one can change, add, delete or adjust modeling parameters to get the best results if needed. Moreover, certain embodiments work with any type of resource types, QoS metrics and SLAs.

Still another advantage may be that certain embodiments reduce the SLA violations and increase the MTBF by providing an optimally overprovisioned initial resource footprint configuration and replication size for each microservice based on the extracted QoS behaviour models and with regards to the given user's SLA constraints.

Yet another advantage may be that certain embodiments may work on either a lab/test environment (and extrapolate the results to predict QoS in production environment/firm and recommend resource with high accuracy), or the production environment/firm).

Still another advantage may be that, even though the disclosed techniques and solutions were applied to cloud-native applications, one can apply the techniques and solutions to any kind of cloud applications.

Yet another advantage may be that certain embodiments effectively reduce the cost of dimensioning through reduced testing and more accurate predictions of initial resource footprint and replication size configuration. Typically, it is too expensive to measure all traffic profiles and dimensional parameters. Therefore, traditionally it's been calculated based on the input parameters from the customer on-demand. However, certain embodiments disclosed herein make it possible to accurately estimate the QoS metrics beforehand and with lower cost. This may be essential when running microservices as the they may have many more dimensional parameters.

Another technical advantage may be that certain embodiments reduce the energy consumption by assigning an efficient capacities and replication sizes to each microservice.

Still another technical advantage may be that certain embodiments are compatible with user's custom configurations (such as custom affinity/anti-affinity rules).

Particular embodiments are described in <FIG> of the drawings, like numerals being used for like and corresponding parts of the various drawings.

Certain embodiments of the present disclosure may provide solutions enabling model-based resource recommendation for cloud applications by the network node. Certain embodiments may include functionality providing a model based approach for extracting resource capacity and replication size versus application's QoS behaviours. In particular, certain embodiments may perform only a minimal set of benchmarks (instead of trying out all possible combinations) to recommend an efficient initial resource configuration for each microservice. According to certain embodiments, the recommended resource configuration may satisfy all the required QoS constraints agreed on a given user's SLA. In a particular embodiment, the techniques may be implemented on a cloud-native <NUM> HSS-frontend prototype together with an MME simulator, which both are deployed on top of Kubernetes. Such a system may reach <NUM>% accuracy in the initial modeling phase.

<FIG> illustrates a high-level sequence diagram <NUM> of the proposed solution, according to certain embodiments. Note that actors, steps and actions in this and other illustrations may be different in various contexts.

At step <NUM>, the sequence begins with the instantiation of the procedure by a system architect, a customer, a vendor, or another user of the system. As depicted, in particular embodiments, the user may instantiate the process by providing resource types, quality metrics, capacities to test, service chains, microservice(s) details, lab environment configurations, affinity/anti-affinity rules, and other parameters.

At step <NUM>, the modeling component of the NFV-Inspector prepares the lab environment and deploys the microservices in the lab environment. As part of the preparation process, the saturation points and minimal test points may be discovered.

At step <NUM>, the benchmarking procedures are performed. As described in more detail below, the objective is to select and benchmark only a small subset of configurations and predict the value of the rest using ML models.

At step <NUM>, the automated testing tool provides QoS measurements. The QoS measurements are used to generate a QoS behaviour model at step <NUM>. In a particular embodiment the QoS behaviour model includes a ML QoS behaviour model.

<FIG> illustrates an example organization of an NFC system <NUM>, according to certain embodiments. As shown in <FIG>, the NFC system is composed of a set of CNFs <MAT> <NUM> in which consists of a number of microservices <MAT> <NUM>. According to certain embodiments, each microservice MSi,j <NUM> may be load-balanced among a set of replication instances <MAT> and placed on a substrate network, which is the underlying physical network. Worker nodes <NUM> that compose network denotes as <MAT>. Each worker node <NUM> can host one or several microservice <NUM> instances. Each mo ∈ M may be specified with its resource types capacities Crd (mo), where rd ∈ R is a resource types (CPU, memory, huge-page table, etc.). In the same way, each microservice instance <MAT> <NUM> capacity is denoted as <MAT> with respect to the minimum and maximum allocatable capacity to each resource type denoted as <MAT> and <MAT> respectively. The total amount of assigned resource capacities of a microservice MSi,j <NUM> is denoted on each resource type rd as follows: <MAT>.

<FIG> illustrates a service chain flow <NUM> in an NFC system. The goal of these microservices <NUM> is to serve a series of service chains <MAT> 310a-z, as depicted in <FIG>. These service chains 310a-z represent the connection and traffic flow between microservices <NUM>, which may be specified as a directed graph G(SCk) = (MSk, Ek), where MSk is a subset of all microservices <NUM> that have a role in providing service SCk and Ek is the subset of all edges belongs to the service chain SCk 310a-z. Each service chain SCk 310a-z has a request entry point that receives requests from potential requesters via the Network Function Virtualization Infrastructure (NFVI). To provide a service Sk, each SCk 310a-z traverses an ordered set of microservices (MSi,j,. , MSi' ,j' ) <NUM> belonging to (CNFi,. These microservices <NUM> are connected through one or more directed virtual links {ek,<NUM>(MSa,b, MSa' ,b' ),. , ek,n(MSc,d, MSc' ,d' )}. Moreover, according to certain embodiments, the NFC system comes with a set of affinity/anti-affinity rules for the placement of the microservices <NUM> in an environment. For example, a pair of microservices <NUM> may need to be placed on the same worker node <NUM> if they have a high affinity relation. A pair of microservices <NUM> would have a high affinity, for example, if the pair communicate frequently with each other and exchange a large amount of data. On the other hand, some microservices <NUM> may need to be placed on separate worker nodes <NUM> if they have a high anti-affinity relation. This can happen, for example, when there exist two critical microservices <NUM> and there is a need to ensure the service availability in the case of failure of a physical node. These criteria are assumed and will be provided by the system admin.

Matrix U and U' are created to hold all the affinity and anti-affinity rules respectively. Each matrix has dimension |MS| × |MS| where each row and column represents one of the microservices. An entry Uab takes on a value of <NUM> if microservices a and b have a high affinity and must be placed at the same worker node. Additionally, an entry <MAT> takes on a value of <NUM> if microservice a and b have a high anti-affinity and must not be placed at the same worker node <NUM>. <MAT> <MAT>.

To evaluate the performance of an NFC system, one should measure the key QoS metrics (serving throughput, latency, etc.). This set of QoS metrics may be defined as <MAT>. Assuming that an SLA agreement for a customer α on service chain SCk 310a-z and QoS metric qc ∈ Q is in place, <MAT> may be defined such that it denotes the guaranteed value of QoS metric qc in user αSLA agreement on service chain SCk 310a-z.

According to certain embodiments, CNFs QoS metrics behaviour modeling may be based on an algorithm that performs a minimal set of benchmarks and recommends a resource allocation and replica size configuration for each microservice <NUM> in a service chain <NUM>. In other words, for each SCk, <NUM> given a set of SLA requirements <MAT>, the final goal is to recommend an optimal amount of resource capacities for each microservice MSi,j <NUM> in a production environment based on a minimum number of experiments in a smaller-scale deployment and guaranty the agreed QoS constraints on all QoS metrics qc ∈ Q for a user α. It is assumed that the NFC system comes with a sophisticated benchmark tool that allows to define a test scenario based on a given service chain(s), arrival rate(s), duration(s) and burst setting(s).

<FIG> illustrates an example modeling process <NUM>, according to certain embodiments. Specifically, at a step <NUM>, an initial placement of microservices <NUM> is determined. At step <NUM>, the saturation points for a set of data are found. At step <NUM>, training data is generated. At step <NUM>, a model is fit based on a hypothesis. Each of these steps are described in more detail below.

For example, with regard to the initial placement of the microservices <NUM> at step <NUM>, it is recognized that before starting to perform benchmarks, the CNF's microservices are deployed on a lab environment. According to certain embodiments, in order to get accurate results, this deployment may follow one or more of the following criteria:.

<FIG> illustrates an example initial placement of a number of microservices 215a-d on worker nodes 220a-d, respectively, according to certain embodiments. Specifically, as depicted each of microservices 215a-d are placed on a respective one of worker nodes 220a-d. It is recognized, however, that <FIG> is merely an example of an initial placement. Multiple microservices <NUM> may be placed on a single worker node <NUM>, in other embodiments.

As mentioned in previous sections, one of the main challenges with CNFs is the huge number of separately configurable microservices <NUM> as well as sheer number of combinations of configuration parameters which makes it very time consuming to benchmark all possible setting. If there is no systemic bottleneck in a deployment (based on the criteria discussed above) and QoS measures in a service chain <NUM> behave harmonically (as is typical) with the amount of resource capacities assigned to its microservices <NUM>, an algorithm is proposed to effectively select and benchmark only a small subset of configurations and predict the value of the rest using machine learning models. Because it is not necessary to try out all possible resource configurations with this methodology, both the dimensioning time as well as the number of benchmarks required is dramatically reduced for modeling the effect of resource configuration of microservices <NUM> on a service chain's QoS measurements.

The goal of certain embodiments is to model the effect of resource allocation configuration (e.g. CPU and memory capacities) of each microservice <NUM> in a service chain <NUM> on overall QoS metric values (e.g. throughput). But, before starting the modeling procedure, a set of data points is required to train the machine learning models, as discussed above with regard to step <NUM> of <FIG>.

According to certain embodiments, such training data is gathered by selecting a single microservice <NUM> at a time and assigning a resource allocation configuration Tp = (tp(rd<NUM>),. , tp (rd|R|)) (e.g. a tp(cpu) can be measured as the milicpu, or for memory can be measured as bytes) to each of its resource types rd ∈ R (e.g., CPU, memory, etc.), while assigning maximum possible resources to other microservices in the service chain <NUM>. In this way and considering the placement of all microservices <NUM> are following all the criteria discussed above, it can be safely assumed that the only microservice <NUM> that can possibly become a bottleneck (before the QoS metrics start saturating) is the one under the test. With this method, QoS metric qc ∈ Q (e.g. serving throughput, latency, CPU utilizations, etc.) values Vk(qc) (e.g., throughput or latency) start saturating after a specific CPU and memory capacity threshold. Even in some cases, assigning more resources (e.g., CPU cores) may cause lower QoS (e.g., less serving throughput) due to possible contentions. If a microservice CPU or memory capacity has a small contribution on the service chain's final QoS, utilizing such benchmarking procedure may result in quick saturation of the QoS metric values. In contrast, if the microservice <NUM> plays a key role in providing the intended service, system's QoS metric values may start saturating on the edge of worker node's CPU capacity limits (or even never saturate). This means that if bigger worker nodes are used and more CPU or memory capacities are assigned to these key microservices <NUM> in the service chain <NUM> or if more replicas/instances of the microservice <NUM> are created, then other less-contributing microservices <NUM> may start saturating slower. Therefore, to form an accurate prediction model, all saturation points may be excluded from the test data. For the rest of the test space, only a random number of points are kept so as to both reduce the number of experiments (and as a result reduce the dimensioning time) as well as to avoid over-fitting of the models.

<FIG> illustrates a minimal benchmarking procedure, according to certain embodiments. At step <NUM>, the saturation point is determined. For example, a binary search may be performed to find the saturation point on one resource type (e.g., memory capacity). The same search may then be repeated for other resource types (e.g., CPU capacity). The process will result in finding the key saturation point. The saturation point corresponds to the point at which increasing resource capacities after that point will not have dramatic effect on the system's performance. As a result, QoS metrics starts saturating after that point.

<FIG> illustrates an example plot <NUM> for determining saturation points, according to certain embodiments. Specifically, <FIG> represents all the attempts to find the saturation point in a Lightweight Directory Access Protocol (LDAP) Server prototype, according to one particular example embodiment.

Returning to <FIG>, after finding this key saturation point, the saturated area associated with a saturation point is removed from the test space, at step <NUM>. For example, the test space may be divided into divisions and all the points in the saturated area may be excluded.

At step <NUM>, random tests are then performed on the non-saturated area. The random tests are performed with random resource capacities. For the rest of the points, a random set of points is selected in the test space and the number of test points is kept proportional to the size of each division to avoid over-fitting of the machine learning models in the next step. <FIG> illustrates an example plot <NUM> depicting the performance of random benchmarks in a non-saturated area, according to certain embodiments.

At step <NUM>, measured QoS values and corresponding capacities are used as input for ML models. More specifically, according to certain embodiments, regression models are used to form QoS behaviour models of an NFV system based on the training data gathered in step <NUM>, so that for a given a resource allocation configuration, the value of a given QoS metric may be predicted when a particular service chain is running. Once this data is collected, for a given microservice MSi,j, QoS metric qc and a service chain SCk a separate regression model <MAT> is trained to predict the value of qc when resource allocation size of MSi,j is Tp.

<FIG> represents the entire data collection and modeling workflow, including hypotheses selection and training procedure. Specifically, at step <NUM>, the training data is obtained. According to a particular embodiment, to train each regression model <MAT> we use the training input set <MAT> that was constructed using the process described above with regard to <FIG> along with corresponding targets <MAT> the former of which represents a multi-dimensional matrix as each Tp itself is a set of resource allocation sizes for all resource types.

At step <NUM>, the hypotheses are trained. For example, to learn an approximation regression model with the objective of making accurate prediction of QoS metric value Vk(qc) for previously unseen template Tp, the disclosed method tries out different hypothesis functions hi(. ) and, at step <NUM>, chooses the one that fits best with the training data such that the selected hypothesis function has the highest Goodness of Fit (GoF) as well as better cross-validation results. Naturally, each microservice <NUM> may behave differently, and therefore different hypotheses may need to be tested. Studies have shown that the throughput of around <NUM>% of all type of applications can be modeled using a polynomial hypotheses. For instance, assuming there are two resource types r<NUM> and r<NUM> (e.g., CPU, memory, etc.) and a quality metric qc (e.g., serving throughput), and assuming all microservices follows a monotonically increasing performance model, such that by increasing the capacity of each resource type (up to a limit) QoS measurements will either increase or remain the same, the hypotheses <MAT> may be defined as (but are not limited to) as follows: <MAT> <MAT> <MAT> <MAT> <MAT>.

After training all hypotheses <MAT> and based on their GoF values and cross validation results, a final hypothesis <MAT> will be selected at step <NUM> and a regression model <MAT> is trained to estimate the value of Vk(qc) when the resource allocation configuration of MSi,j is equal to Tp.

<FIG> depicts another example method for performance modeling of a plurality of microservices. At step <NUM>, the method begins when the plurality of microservices are deployed within a network. The plurality of microservices are communicatively coupled to generate at least one service chain for providing at least one service. At step <NUM>, an initial set of training data is determined for the plurality of microservice based on a resource allocation configuration. At step <NUM>, at least a portion of data is excluded from the initial set of training data to generate a subset of training data. At step <NUM>, a QoS behaviour model is generated based on the subset of the training data.

Excluding the portion of data from the initial set of training data to generate the subset of training data includes: isolating resource allocation to each of the plurality of microservices; selecting one of the plurality of microservices; assigning a maximum respective resource allocation configuration to each of a plurality of resources associated with the microservices; determining a saturation point for the one of the plurality of microservices; and excluding a saturation area associated with the saturation point from the initial set of training data when generating the subset of training data.

In a particular embodiment, the saturation point includes a point when a quality of service associated with the one of the plurality of microservices starts saturating while increasing resources.

In a particular embodiment, the method further includes repeating the steps of isolating, selecting, assigning, determining, and excluding for a randomly selected subset of the plurality of microservices.

In a particular embodiment, the environment comprises a lab environment, which may include, as one example, a performance testing environment. The lab environment includes at least one infrastructure element. The at least one infrastructure element of the lab environment is selected to emulate a production environment to minimize prediction errors.

In a particular embodiment, the microservices may be isolated during placement, at a host level. If that is not possible, the microservices may be pinned to a CPU. For example, Kubernetes allows a docker process to be pinned to a particular set of cores. This may be done to avoid the nosy neighbor problem.

In a particular embodiment, a network topology of the lab environment emulates a network topology of the production environment.

In a particular embodiment, the at least one infrastructure element includes at least one of equipment, hardware, operating system, and bandwidth selected to emulate the production environment.

In a particular embodiment, deploying the plurality of microservices within the network includes using a network emulator.

In a particular embodiment, deploying the plurality of microservices within the network comprises determining at least one network node to host each of the plurality of microservices.

In a particular embodiment, a communication capacity of each of the plurality of microservices is considered when determining the at least one network node to host each of the plurality of microservices. In particular embodiments, the communication capacity may be the network bandwidth.

In a particular embodiment, the plurality of microservices are hosted on a plurality of network nodes. Each of the plurality of network nodes is elected to host at least one of the plurality of microservices based on at least one affinity rule or randomly.

In a particular embodiment, deploying the plurality of microservices within the network includes minimizing factors affecting QoS such that the resource allocation configuration is the only deterrent to getting a higher QoS value.

In a particular embodiment, selecting the QoS behaviour model based on the set of training data includes: testing a plurality of hypothesis functions for the plurality of microservices in the service chain; selecting one of a plurality of hypothesis functions that has a highest goodness of fit and cross-validation results to the set of training data; and training the QoS behaviour model to estimate at least one QoS metric. Though the steps described herein include training the QoS behoviour model to estimate at least one QoS metric, it is recognized that the techinques may be used to estimate any other other kind of QoS metric, resource type, and/or hypothesis function.

In a particular embodiment, the method further includes using the QoS behaviour models to determine a value of the at least one QoS metric for the plurality of microservices in each service chain.

In a particular embodiment, the method further includes using the QoS behaviour models to determine an optimal amount of resource capacities of each of the plurality of microservices belonging to the service chain.

In a particular embodiment, the QoS behaviour model is a ML model.

<FIG> illustrates a schematic block diagram of a virtual apparatus <NUM> in a network. The apparatus may be implemented in a wireless device or network node. Apparatus <NUM> is operable to carry out the example method described with reference to <FIG> and possibly any other processes or methods disclosed herein. It is also to be understood that the method of <FIG> is not necessarily carried out solely by apparatus <NUM>. At least some operations of the method can be performed by one or more other entities.

Virtual Apparatus <NUM> may comprise processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and/or data communications protocols as well as instructions for carrying out one or more of the techniques described herein, in several embodiments. In some implementations, the processing circuitry may be used to cause deploying module <NUM>, determining module <NUM>, excluding module <NUM>, selecting module <NUM>, and any other suitable units of apparatus <NUM> to perform corresponding functions according one or more embodiments of the present disclosure.

According to certain embodiments, deploying module <NUM> may perform certain of the deploying functions of the apparatus <NUM>. For example, deploying module <NUM> may deploy a plurality of microservices within a network. The plurality of microservices are communicatively coupled to generate at least one service chain for providing at least one service.

According to certain embodiments, determining module <NUM> may perform certain of the determining functions of the apparatus <NUM>. For example, determining module <NUM> may determine an initial set of training data for the plurality of microservice based on a resource allocation configuration.

According to certain embodiments, excluding module <NUM> may perform certain of the excluding functions of the apparatus <NUM>. For example, excluding module <NUM> may exclude at least a portion of data from the initial set of training data to generate a subset of training data.

According to certain embodiments, generating module <NUM> may perform certain of the generating functions of the apparatus <NUM>. For example, generating module <NUM> may generate a QoS behaviour model based on the subset of the training data.

<FIG> illustrates a wireless network in which the above described methods for performance modeling of a plurality of microservices may be implemented, in accordance with some embodiments. Although the subject matter described herein may be implemented in any appropriate type of system using any suitable components, the embodiments disclosed herein are described in relation to a wireless network, such as the example wireless network illustrated in <FIG>. For simplicity, the wireless network of <FIG> only depicts network <NUM>, network nodes <NUM> and 1260b, and wireless devices <NUM>, 1210b, and 1210c. In practice, a wireless network may further include any additional elements suitable to support communication between wireless devices or between a wireless device and another communication device, such as a landline telephone, a service provider, or any other network node or end device. Of the illustrated components, network node <NUM> and wireless device <NUM> are depicted with additional detail. The wireless network may provide communication and other types of services to one or more wireless devices to facilitate the wireless devices' access to and/or use of the services provided by, or via, the wireless network.

Network node <NUM> and wireless device <NUM> comprise various components described in more detail below.

<FIG> illustrates an example network node <NUM>, according to certain embodiments.

Interface <NUM> is used in the wired or wireless communication of signalling and/or data between network node <NUM>, network <NUM>, and/or WIRELESS DEVICEs <NUM>. Radio front end circuitry <NUM> may receive digital data that is to be sent out to other network nodes or WIRELESS DEVICEs via a wireless connection.

In some embodiments, some signaling can be affected with the use of control system <NUM> which may alternatively be used for communication between the hardware nodes <NUM> and radio units <NUM>.

The components of the systems and apparatuses may be integrated or separated. Moreover, the operations of the systems and apparatuses may be performed by more, fewer, or other components. Additionally, operations of the systems and apparatuses may be performed using any suitable logic comprising software, hardware, and/or other logic. As used in this document, "each" refers to each member of a set or each member of a subset of a set.

The invention and preferred embodiments are defined by the following claims.

Claim 1:
A method (<NUM>) for performance modeling of a plurality of microservices (<NUM>), the comprising:
deploying the plurality of microservices (<NUM>) within a network (<NUM>), the plurality of microservices (<NUM>) communicatively coupled to generate at least one service chain (<NUM>) for providing at least one service;
based on a resource allocation configuration, determining an initial set of training data for the plurality of microservices within the network (<NUM>), the method being characterised by:
excluding at least a portion of data from the initial set of training data to generate a subset of training data, wherein excluding the portion of data from the initial set of training data to generate the subset of training data comprises:
isolating resources of the plurality of microservices;
selecting one of the plurality of microservices;
assigning a maximum respective resource allocation configuration to each of a plurality of resources associated with the plurality of microservices;
determining a saturation point for the one of the plurality of microservices; and
excluding a saturation area associated with the saturation point from the initial set of training data when generating the subset of training data; and
generating a Quality of Service, QoS, behaviour model based on the subset of the training data.