Patent Description:
With the fifth generation of mobile technology (<NUM>), wireless networks increase their capabilities in terms of achieved throughput (enhanced Mobile Broadband, eMBB), latency (Ultra-reliable low latency communication, URLLC), and device density (massive Machine Type Communications, mMTC). But, besides that, it provides improved flexibility to apply these new capabilities to a specific subset of devices, using approaches such as network slicing or non-public networks (NPNs). This opens a new set of video-related services for professional users and vertical industries which, in previous technological generations, were only possible using wired networks, ad-hoc wireless links, or even were not possible whatsoever.

Addressing this set of use cases is challenging for several reasons. Their requirements and Quality of Experience (QoE) expectations may be different from the ones typically present in "traditional" video communication services, such as streaming or videoconference. The experience and expectations of the use case owners may not be applicable to cellular wireless networks, even when QoS policies are applied. Furthermore, professional and vertical markets typically have much less users than the video consumer market (there are fewer content producers than content consumers), or the video transmission is just one of the pieces of a much more complex ecosystem (as e.g. in the automotive industry). As a consequence of that, totally new use cases are being addressed with the same set of coding and QoE evaluation tools that were being used for entertainment and personal communication video.

When designing mission-critical services, such as tele-operated driving (ToD), the use case may normally assume that the bitrate for transferring video is high enough so that the video quality is high for all operators (drivers) in all conditions. It may also be assumed that some level of traffic prioritization (QoS guarantee) is available, e.g. non-public networks or slices.

For instance, the <NUM> Automotive Association (5GAA) is setting a requirement of <NUM> Mbps per HD camera (in H. <NUM>), for a total of <NUM> Mbps of uplink. This way it may be possible to guarantee sufficient visual quality for all the possible situations. However, when the codec is not stressed (e.g. when driving at low speeds in static scenarios), this may be a waste of bandwidth in the network.

Document "<NPL>", describes perceptual adaptive quantization techniques based on deep neural network and high efficiency video coding (HEVC) for bitrate reduction. Document <CIT> discloses a system and method for process video based on quantization parameters used to encode image blocks of the video.

Therefore, one problem is how to reduce the bitrate when possible, without affecting the visual quality and therefore the service.

Some embodiments provide a method, apparatus, and computer program product for controlling transmission of small amount of data from a user equipment to a wireless network when the user equipment is in an inactive state without changing the state of the user equipment from inactive state to connected state.

In particular, some embodiments provide at least one of the following two key elements:.

Both the above-mentioned elements can be used separately or combined, but using both of them together may achieve more efficient encoding than when using only one of them.

According to some embodiments training is done for a specific subject and algorithm(s) is/are targeted only to an encoder to be able to keep constant perceptual quality for a mission-critical user. The encoder works in a frame-by-frame basis, thus aiming to obtain lowest latency for mission-critical applications.

The equipment can be inserted in a loop of a coding processes (QP decision, intermediate PSNR calculation).

According to an embodiment, the rate-control outputs a quantization parameter (QP) per each frame.

According to an embodiment of an algorithm, supervised training is used, since the embodiment is targeting for a single user and a single task, where it is assumed that the diversity of video sources is limited. This may provide much better performance than a generic reinforcement-learning based approach like a Video Quality Awareness Rate Control for real-time video streaming via deep reinforcement learning (QARC).

Decisions are based on the frame to encode, normalized through a pre-defined QP which has been selected to optimize the coding efficiency.

Temporal correlation in the video is represented by the quantization parameter QP and normalization Peak Signal to Noise Ratio (PSNR), which are later used for training and inference in a neural network. This may allow a relatively shallow convolutional neural network (CNN) to be used for the purpose.

According to some aspects, there is provided the subject matter of the independent claims. Some further aspects are defined in the dependent claims. The embodiments that do not fall under the scope of the claims are to be interpreted as examples useful for understanding the disclosure.

According to a first aspect there is provided a method comprising:.

According to a second aspect there is provided an apparatus comprising:.

According to a third aspect there is provided an apparatus comprising at least one processor; and at least one memory including computer program code the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:.

According to a fourth aspect there is provided a computer program comprising computer readable program code which, when executed by at least one processor; cause the apparatus to perform at least the following:.

Neural networks (NN) are being utilized in an ever increasing number of applications for many different types of device, such as mobile phones. Examples include image and video analysis and processing, social media data analysis, device usage data analysis, etc..

A property of neural nets (and other machine learning tools) is that they are able to learn properties from input data, for example in supervised way or in unsupervised way. Such learning may be a result of a training algorithm, or of a meta-level neural network providing the training signal.

A neural network (NN) is a computation graph comprising several layers of computation. Each layer comprises one or more units, where each unit performs an elementary computation. A unit is connected to one or more other units, and the connection may have associated a weight. The weight may be used for scaling the signal passing through the associated connection. Weights are usually learnable parameters, i.e., values which can be learned from training data. There may be other learnable parameters, such as those of batch-normalization layers.

Two example architectures for neural networks are feed-forward and recurrent architectures. Feed-forward neural networks are such that there is no feedback loop: each layer takes input from one or more of the preceding layers and provides its output as the input for one or more of the subsequent layers. Also, units inside a certain layer take input from units in one or more of preceding layers, and provide output to one or more of following layers.

Initial layers (those close to the input data) extract semantically low-level features such as edges and textures in images, and intermediate and final layers extract more high-level features. After the feature extraction layers there may be one or more layers performing a certain task, such as classification, semantic segmentation, object detection, denoising, style transfer, super-resolution, etc. In recurrent neural nets, there is a feedback loop, so that the network becomes stateful, i.e., it is able to memorize information or a state.

The input layer receives the input data, such as images, and the output layer is task-specific and outputs an estimate of the desired data, for example a vector whose values represent a class distribution in the case of image classification. The "quality" of the neural network's output is evaluated by comparing it to ground-truth output data. The comparison may include a loss or cost function, run on the neural network's output and the ground-truth data. This comparison would then provide a "loss" or "cost" value.

The weights of the connections represent the biggest part of the learnable parameters of a neural network. Hereinafter, the terms "model", "neural network", "neural net" and "network" are used interchangeably, as well as the weights of neural networks are sometimes referred to as learnable parameters or simply as parameters. It is noted, for the sake of clarity, that the weights of neural networks referred to as parameters are different parameters than e.g. coding parameters of video codecs referred to above.

In general, the training algorithm may include changing some properties of the neural network so that its output is as close as possible to a desired output. For example, in the case of classification of objects in images, the output of the neural network can be used to derive a class or category index which indicates the class or category that the object in the input image belongs to. Training may include minimizing or decreasing the output's error, also referred to as the loss. Examples of losses are mean squared error, cross-entropy, etc. In deep learning techniques, training may be an iterative process, where at each iteration the algorithm modifies the weights of the neural net to make a gradual improvement of the network's output, i.e., to gradually decrease the loss.

The parameters are learned by means of a training algorithm, where the goal is to minimize the loss value on a training dataset and on a held-out validation dataset. In order to minimize such value, the network is run on a training dataset, a loss value is computed for the whole training dataset or for part of it, and the learnable parameters are modified in order to minimize the loss value on the training dataset. However, the performance of the training is evaluated on the held-out validation dataset. The training dataset is regarded as a representative sample of the whole data. One popular learning approach is based on iterative local methods, where the loss on the training dataset is minimized by following the negative gradient direction. Here, the gradient is understood to be the gradient of the loss with respect to the learnable parameters of the neural network. The loss may be represented by the reconstructed prediction error. Computing the gradient on the whole training dataset may be computationally too heavy, thus learning is performed in sub-steps, where at each step a mini-batch of data is sampled and gradients are computed from the mini-batch. This is referred to as stochastic gradient descent. The gradients are usually computed by back-propagation algorithm, where errors are propagated from the output layer to the input layer, by using the chain rule for differentiation. If the loss function or some operations performed by the neural network are not differentiable, it is still possible to estimate the gradient of the loss by using policy gradient methods, such as those used in reinforcement learning. The computed gradients are then used by one of the available optimization routines (such as stochastic gradient descent, Adam, RMSprop, etc.), to compute a weight update, which is then applied to update the weights of the network. After a full pass over the training dataset, the process is repeated several times until a convergence criterion is met, usually a generalization criterion. A generalization criterion may be derived from the loss value on the held-out validation dataset, for example by stopping the training when the loss value on the held-out validation dataset is less than a certain threshold. The gradients of the loss, i.e., the gradients of the reconstructed prediction error with respect to the weights of the neural network, may be referred to as the training signal.

In the context of training neural networks, an epoch refers to the time during which a neural network is used on all the training data, and it may comprise a plurality of iterations. An iteration refers to the time during which a neural network is used on a subset of the whole training data, typically referred to as a batch or mini-batch of data. Typically, an update to the weights of the neural network is performed at each iteration. In particular, a batch of data is input to the network, an output is obtained from the network, a loss is computed from the output, gradients of the loss with respect to the weights are computed, and weight updates are determined based on the computed gradients.

Training a neural network is an optimization process, but as a difference to a typical optimization where the only goal is to minimize a function, the goal of the optimization or training process in machine learning is to make the model to learn the properties of the data distribution. In other words, the goal is to learn to generalize to previously unseen data, i.e., data which was not used for training the model. This is usually referred to as generalization. In practice, data may be split into two (or more) sets, the training set and the validation set. The training set is used for training the network, i.e., to modify its learnable parameters to minimize the loss. The validation set is used for checking the performance of the network on data which was not used to minimize the loss, as an indication of the final performance of the model. In particular, the errors on the training set and on the validation set are monitored during the training process to understand the following issues:.

If the network is learning at all - in this case, the training set error should decrease, otherwise we are in the regime of under fitting.

If the network is learning to generalize - in this case, also the validation set error needs to decrease and to be not too much higher than the training set error. If the training set error is low, but the validation set error is much higher than the training set error, or it does not decrease, or it even increases, the model is in the regime of overfitting. This means that the model has just memorized the training set's properties and performs well only on that set, but performs poorly on a set not used for tuning its parameters.

Neural networks may be used for compressing and de-compressing data, such as video/images in connection, for example, with video-related services. Such neural networks are usually trained to minimize a combination of bitrate and distortion, where the distortion may be measured by Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM) index, or similar, computed on the decoded data and the original data.

An example of new video-related services is Tele-operated Driving (ToD). Tele-operated driving can be seen as a side effect of tackling with potential issues that autonomous driving cannot solve by themselves. In this case human intervention might be required to drive the car. Depending on the level of implication of the remote operator in the act of driving, different ToD types may be defined. Table <NUM> below shows examples of the role and engagement of the ToD operator in the act of driving in different types of ToD. This example is provided by the <NUM> Automotive Association (5GAA) in the document "Tele-Operated Driving (ToD): Business Considerations, published on <NUM> July <NUM>. It should be noted that tele-operated driving is only one example where embodiments of the disclosure may be utilized.

Different ToD types are defined based on the act of driving. Examples of the ToD types are presented below in Table <NUM>.

In the following, some explanations of the terms used in Table <NUM> will be shortly provided.

In Type <NUM> the ToD operator is not engaged in the act of driving. In other words, the ToD is taking no role in the act of driving. All three levels of driving operations, i.e. Strategic level, Tactical level, and Real-Time Operational and Real-time Tactical level, are performed by an in-vehicle user or system such as a driving automation system. In this case the ToD operator may monitor the status of the vehicle and send information to the in-vehicle user or system supporting the act of driving.

In Type <NUM> the ToD operator takes on the role of Dispatcher, which is only to perform the Strategic level operations of driving, e.g. travel planning, route and itinerary selection, while the Tactical and Operational level operations are performed by the in-vehicle user or system.

In Type <NUM> the ToD operator takes the role of Indirect Controller (Remote Assistant), to perform the Tactical level functions like pathway planning, which corresponds to the remote assistance function defined for driving automation systems. If needed, the Indirect Controller may also perform Strategic level operations of driving. In this type real-time Operational level and real-time Tactical level functions, i.e. Dynamic Driving Task (DDT), are performed by in-vehicle user or system. When engaged in the act of driving, the remote operator of Indirect Control ToD may disengage the in-vehicle system from performing DDT, by either taking over all DDT tasks, i.e., the role of Direct Controller, or by bringing the vehicle to a minimal risk condition. When Indirect Control ToD is engaged, the ToD operator may also perform Strategic level operations such as reselecting the route, when such operations are needed to complete the act of driving, e.g. to avoid a blocked road.

In Type <NUM> the ToD operator takes the role of Direct Controller (Remote Driver), to perform all or part of real-time operational and real-time tactical functions (i.e. DDT), which corresponds to the remote driving function defined for driving automation systems. If needed, the Direct Controller may also perform Tactical and Strategic level operations of driving. When Direct Control ToD is engaged, part of the DDT functions, e.g. lateral and/or longitudinal vehicle motion control, may be performed by the In-vehicle user or system, e.g. through adaptive cruise control and/or lane keeping, while the ToD operator is still responsible for the OEDR task. When Direct Control ToD is engaged, the ToD operator may also perform Strategic level operations such as reselecting the route and Tactical level operations such as replanning the pathway, when such operations are needed to complete the act of driving, e.g. to avoid a blocked road or get around an obstacle in the road.

For the purposes of this disclosure, the types under consideration are the ones where part of or all of the dynamic driving tasks are performed by a remote driver on a sustained basis. Using the definitions of Table <NUM> these types are ToD Type <NUM> or <NUM>.

Although several implementations of the ToD service are possible, a typical reference architecture assumes that there are four cameras boarded on the vehicle, one at each side i.e. in the front, in the back, in the left side and in the right side of the vehicle, as well as other sensors (positioning system, radar, lidar, etc.). Information coming from those sources may be processed by an On-Board Unit (OBU) and sent through a <NUM> network to a remote driver cockpit. The cockpit may have several screens, or maybe a virtual environment using virtual reality (VR) equipment (e.g. VR glasses), where the video from the cameras is displayed, as well as the information from the sensors. Besides, the cockpit contains remote control (e.g. a driving wheel, a throttle) whose information is also sent back to the car.

<FIG> illustrates an example architecture of a ToD system. The host vehicle (HV) is the object of the ToD operations performed by a remote driver (RD). Communication between the host vehicle and the remote driver may be performed via a wireless communication network (NW) such as a <NUM> network.

In a so-called mission-critical tasks where latency i.e. the time between transmission of a signal by a transmitter and reception of the signal by a receiver should not exceed a value defined for such mission-critical task, the coding system should try to ensure that a minimum level of quality is always guaranteed. However, lowering bitrate while keeping perceptual quality may not be a straightforward task when dealing with mission-critical tasks.

In order of efficiency, several strategies for lowering bitrate may be possible.

One strategy is performing compressing using constant quality instead of constant bitrate. As an example, instead of using a certain bitrate such as <NUM> Mbps, a certain quantization parameter (QP) such as QP ≈ <NUM> is used. The quantization parameter is related to perceptual quality, but this relation is not perfect. Therefore, to achieve a good perceptual quality in all situations, a low QP should be selected (i.e. a safety margin may still be needed).

Another strategy is using perceptually-constant quality compression. For example, instead of using a certain quantization parameter value such as QP ≈ <NUM>, the data to be transmitted is compressed so that a so-called Mean Opinion Score (MOS) is higher than or equal to a predetermined value (e. g MOS ≥ <NUM>).

The MOS can be calculated as <MAT> where ui,j,r is the observed rating for subject I, a processed video sequence j, repetition r and N is the number of subjects.

Even though the latter strategy may be better than constant quantization parameter, it is worth noting that the Mean Opinion Score is the opinion of a "mean user". A high level of MOS may need to be selected to satisfy even the most critical users.

Still another strategy is to use per-user perceptual model. Instead of defining a lower limit to the MOS, a personalized encoding may be created for each user so that a User Opinion Score (UOS) is higher than or equal to a predetermined limit. For example, UOS ≥ <NUM> may be defined.

The above-mentioned Mean Opinion Score is a measure, which can be used when estimating quality of experience, representing overall quality of a stimulus or system. According to one definition, MOS is the arithmetic mean over all individual "values on a predefined scale that a subject assigns to his opinion of the performance of a system quality". Such ratings may usually be gathered in a subjective quality evaluation test, but they can also be algorithmically estimated. The MOS may be expressed as a single rational number, for example in the range <NUM>-<NUM>, where <NUM> is the lowest perceived quality, and <NUM> is the highest perceived quality. However, other rating scales may also be used and it may also be possible to use a more dense scaling, such as <NUM>, <NUM>, <NUM>, <NUM>,. , <NUM>, <NUM>.

<FIG> shows a high-level diagram of an embodiment, in which raw video <NUM> is input to a compression block <NUM> to produce compressed video <NUM>. The raw video is also input to a rate control block <NUM> which also receives from a per-user MOS model database <NUM> MOS model which has been generated for that particular user who is performing the tele-operated driving tasks with the host vehicle. The rate control block <NUM> uses the retrieved per-user MOS model to determine an appropriate value for the QP parameter so that the desired MOS can be achieved when compressing the raw video.

<FIG> illustrates some example curves of possible bitrate variations with different rate-control approaches: constant bitrate, constant QP, constant MOS and constant UOS. It should be noted that these curves are illustrative only and different savings compared to the constant bitrate approach may be achieved in different situations.

In the following, some details of a perceptual-quality optimized rate control will be described, in accordance with an embodiment.

At a data collection step data for training an apparatus is collected using representative original content sequences (SRC) from a database as inputs to a training phase. The apparatus is trained to be effective for a known client device, characterized by an optimal resolution, at which some metrics are calculated.

<FIG> illustrates the data collection process, in accordance with an embodiment. The original content sequences have been compressed by a compressing block <NUM> with different quantization parameter values QPn to obtain processed video sequences (PVS), which may be stored <NUM> to a memory attached with information of the video sequence and the quantization parameter used in the compressing. The processed video sequences are provided to a first evaluator <NUM> and a second evaluator <NUM>. The first evaluator <NUM> performs for each frame a so-called fast objective metric evaluation such as a Peak Signal to Noise Ratio (PSNR) evaluation related to encoding with a fixed QP to obtain objective evaluation values PSNR(SRC, PVS, F) for the frames. The second evaluator <NUM> evaluates for each video sequence a subjective score such as MOS related to encoding with a fixed QP. This may be achieved with subjective assessment tests using ITU-T P. <NUM> or a similar methodology, for example.

The minimum QP value (QPtarget) required to reach a target MOS (MOStarget) is derived from the subjective scores.

<FIG> shows a relationship between PVS, PSNR, MOS and QP, and MOS1 selected as MOStarget as an example of determination of the minimum QP value to reach the target MOS.

Next, an example of the training phase will be described.

An aim of the training is to enable a neural network to infer a target value for the quantization parameter QPtarget, which may be used to encode the frame, that will give a MOStarget objective quality, given the following:.

The inputs of the training are the frame to be encoded (SRC Frame), a normalization QP used as a hyperparameter, and an objective metric evaluation value (e.g. PSNR) of the current frame encoded using the normalization QP. The encoded sequence (PVS) calculated at the optimal resolution is compared against the original sequence (SRC).

The training produces the target QP value QPtarget as the output. That relates to the expected MOStarget and represents the expected quality for that image.

The training process is represented in <FIG>. The training process is performed for each source video sequence SRC used for the training. One target MOS value MOStarget have been selected for the training.

For each image the training comprises extracting features with a convolutional neural network (CNN) and training the PSNR (for the image compressed with QPn) and QPn.

The training also comprises tuning the hyperparameter so that the best QP for normalization (QPN) may be selected in evaluation among all QPn values.

After training an inference process may be performed. An example of the inference process is depicted in <FIG>. For each source frame to be encoded, features are extracted by the convolutional neural network. The features to be extracted may be indicative of perceptual quality of the frames, complexity of the frame (i.e. contents of the visual information of the frame) in view of compression efficiency. For example, the features may indicate that the frame has a large number of details which may reduce efficiency of the compression or may require coarser quantization of pixel values of the frame, whereas if the visual information of the frame has much less details the compression may be performed more efficiently than with a frame with more details. Some features may relate to spatial properties of the frame, temporal properties of a sequence of frames etc. A normalization QP (QPN) to execute PSNR values, and expected MOS after encoding i.e. the MOStarget, are also used as inputs of the inference process together with the source frame to be encoded (SRC Frame). The source frame is encoded with the normalization QP and the PSNR is computed, which is provided together with the corresponding QP value to dense layers of the neural network. Also the expected MOS value MOSx is provided to the dense layers. As a result of the inference, the neural network outputs an optimal target value of the quantization parameter QPtarget at which the image should be encoded to get the expected MOStarget.

The system utilizes computation of convolutional neural networks (not too deep). This can be implemented in real-time in platforms embeddable e.g. in cars or similar systems.

As the sequence characteristics do not change abruptly, computation can be done once for each N frames, lowering requirements, with minimum performance loss. For example, using QP cascading, the method may be applied to "key" frames (e.g. the lowest temporal sublayer) and heuristic QP cascading may be applied for the other frames of the hierarchy.

Real-time computation can be done with less than one frame of delay. If computation is done from a previous frame, no latency may be introduced.

If the apparatus, such as a tele-operated vehicle, computes only the PSNR, QP computation can even be done on a cloud implemented in a communication network, e.g. as a cloud service.

In the following, some details of implementing a per-user opinion model with the above described training and inference processes will be described.

According to a subjective experiment, each user who took part the experiment, repeated the same subjective test <NUM> times in different days. Based on the results of the experiment, a user opinion score (UOS) was computed for each user as an average of these ten repetitions.

<FIG> shows the UOS of <NUM> of the users (Observer A to Observer J) that took place in the experiment, and its comparison with the global MOS. The x-axis (HRC) shows the coding bitrate of the videos (labeled as BR1 to BR5, plus the original uncompressed SRC).

It can be concluded from the results that UOS can be significantly different from MOS.

The UOS can be approximated by: <MAT> where Uijr is a random variable representing the subjective scoring of a user i of a processed video sequence j in the repetition r, E[·] represents the expected value of the random variable across a series of repetitions, ψj is the MOS of PVS j and Δi is the bias of a user i, computed as the mean difference between the user scores and the MOS.

Based on this analysis, the UOS can be applied to the algorithm presented above in the following way.

A set of video sequences which are representative of all the conditions of a case is obtained. These video sequences may be obtained, for example, by a car having one or more cameras (a camera car) and capturing videos in different situations. These video sequences are compressed with all QPs selected for the training. The sequences are evaluated once by a pool of representative users to obtain MOS using e.g. a standard methodology (e.g. ITU-T P. The number of users M may be, for example, <NUM>-<NUM> observers (subjects) but also other amount of observers may be used.

Based on this test, the UOS may be obtained in an "approximate" way or in an "exact" way.

In the approximate way each of the observers performs the subjective test once and a global user bias is obtained from results from all observers. The UOS is the MOS ψj plus the bias Δi:
UOS = MOS + BIAS, i.e. <MAT>.

In the exact way each of the observers performs the subjective test R times (e.g. R=<NUM>). Then the UOS is computed as <MAT> i.e. when R=<NUM>, the equation becomes <MAT>.

When the UOS has been calculated it is used instead of MOS to train the rate control algorithm, as described above.

According to some evaluations, when using UOS instead of MOS may allow to target <NUM> MOS points less that with MOS. Negatively-biased users at e.g. UOS=<NUM> may use the same bitrate as MOS=<NUM> thus there will be no bitrate savings for them. Neutral users at UOS=<NUM> may save <NUM>% of bitrate with respect to using MOS=<NUM>. Positively-biased users at UOS=<NUM> may save <NUM>% of bitrate with respect to using MOS=<NUM>.

These are example figures extrapolated from some experiments. Actual numbers will depend on the use case, and they may vary.

For each user, there is a trained rate control which may be loaded in the vehicle so that the user preferences will be taken into account in ToD.

However, the approach is general and can be used to train other metrics and therefore use UOS with other state-of-the art Rate Controllers, with similar gains.

The following describes in further detail a suitable apparatus and possible mechanisms for running a neural network according to embodiments. <FIG> shows an example block diagram of an apparatus <NUM>. The apparatus may be a driving automation system implemented in a vehicle for autonomous driving or it may be another apparatus capable of transmitting video data. As a further example, the apparatus may be an Internet of Things (IoT) apparatus configured to perform various functions, such as for example, gathering information by one or more sensors, receiving or transmitting information, analyzing information gathered or received by the apparatus, or the like. The apparatus may comprise a video coding system, which may incorporate a codec. The apparatus <NUM> may be, for example, a part of the tele-operated vehicle, such as the on-board unit of <FIG>.

The apparatus <NUM> comprises a processor <NUM>, a memory <NUM> and a transceiver <NUM>. The processor is operatively connected to the transceiver for controlling the transceiver. The memory may be operatively connected to the processor <NUM>. It should be appreciated that the memory <NUM> may be a separate memory or included to the processor and/or the transceiver. The memory <NUM> may be used to store information, for example, about maximum length, allowed causes, default values for some parameters and/or for some other information.

<FIG> also illustrates the operational units as a computer code stored in the memory, but they may also be implemented using hardware components or as a mixture of computer code and hardware components.

<FIG> shows the CNN <NUM>, dense layers <NUM> and the regressor <NUM> as computer code blocks, but their implementation may also comprise hardware. The memory <NUM> is also showing some parameters like PVS <NUM>, PSNR <NUM>, per-user generated MOS <NUM> and QP <NUM> in <FIG> as well as a memory area <NUM> reserved for compressed video for transmission.

According to an embodiment, the processor is configured to control the transceiver and/or to perform one or more functionalities described with a method according to an embodiment. Inter alia, the processor <NUM> is configured to execute the convolutional neural network at least when the tele-operated driving operations are in use. The processor <NUM> may also be configured to execute the convolutional neural network at the training phase.

A memory may be a computer readable medium that may be non-transitory. The data processors may be of any type suitable to the local technical environment, and may include one or more of general-purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multi-core processor architecture, as non-limiting examples.

In the following, an example of an access architecture is described with reference to <FIG>. The radio access architecture may be based on Long Term Evolution Advanced (LTE Advanced, LTE-A) or new radio (NR, <NUM>), without restricting the embodiments to such an architecture, however. It is obvious for a person skilled in the art that the embodiments may also be applied to other kinds of communications networks having suitable means by communicating with the tele-operated driving service. Some examples of other options for suitable systems are the universal mobile telecommunications system (UMTS) radio access network (UTRAN or E-UTRAN), long term evolution (LTE, the same as E-UTRA), wireless local area network (WLAN or WiFi), worldwide interoperability for microwave access (WiMAX), Bluetooth®, personal communications services (PCS), ZigBee®, wideband code division multiple access (WCDMA), systems using ultra-wideband (UWB) technology, sensor networks, mobile ad-hoc networks (MANETs) and Internet protocol multimedia subsystems (IMS) or any combination thereof.

<FIG> shows user equipments 110a and 110b configured to be in a wireless connection on one or more communication channels in a cell with an access node (such as (e/g)NodeB) <NUM> providing the cell. The physical link from a user equipment to a (e/g)NodeB is called uplink or reverse link and the physical link from the (e/g)NodeB to the user equipment is called downlink or forward link. It should be appreciated that (e/g)NodeBs or their functionalities may be implemented by using any node, host, server or access point etc. entity suitable for such a usage.

A communication system typically comprises more than one (e/g)NodeB in which case the (e/g)NodeBs may also be configured to communicate with one another over links, wired or wireless, designed for the purpose. These links may be used for signaling purposes. The (e/g)NodeB is a computing device configured to control the radio resources of communication system it is coupled to. The NodeB may also be referred to as a base station, an access point or any other type of interfacing device including a relay station capable of operating in a wireless environment. The (e/g)NodeB includes or is coupled to transceivers. From the transceivers of the (e/g)NodeB, a connection is provided to an antenna unit that establishes bi-directional radio links to user equipments. The antenna unit may comprise a plurality of antennas or antenna elements. The (e/g)NodeB is further connected to core network <NUM> (CN or next generation core NGC). Depending on the system, the counterpart on the CN side can be a serving gateway (S-GW, routing and forwarding user data packets), packet data network gateway (P-GW), for providing connectivity of user equipments (UEs) to external packet data networks, or mobile management entity (MME), etc. The CN may comprise network entities or nodes that may be referred to management entities. Examples of the network entities comprise at least an Access management Function (AMF).

The user equipment (also called a user device, a user terminal, a terminal device, a wireless device, a mobile station (MS) etc.) illustrates one type of an apparatus to which resources on the air interface are allocated and assigned, and thus any feature described herein with a user equipment may be implemented with a corresponding network apparatus, such as a relay node, an eNB, and an gNB.

The user equipment typically refers to a portable computing device that includes wireless mobile communication devices operating with or without a subscriber identification module (SIM), including, but not limited to, the following types of devices: a mobile station (mobile phone), smartphone, personal digital assistant (PDA), handset, device using a wireless modem (alarm or measurement device, etc.), laptop and/or touch screen computer, tablet, game console, notebook, and multimedia device. It should be appreciated that a user equipment may also be a nearly exclusive uplink only device, of which an example is a camera or video camera loading images or video clips to a network. A user equipment may also be a device having capability to operate in Internet of Things (IoT) network which is a scenario in which objects are provided with the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. The user equipment may also utilize cloud. In some applications, a user equipment may comprise a small portable device with radio parts (such as a watch, earphones or eyeglasses) and the computation is carried out in the cloud. The user equipment (or in some embodiments a layer <NUM> relay node) is configured to perform one or more of user equipment functionalities. The user equipment may also be called a subscriber unit, mobile station, remote terminal, access terminal, user terminal or user equipment (UE) just to mention but a few names or apparatuses.

It should also be understood that the distribution of labor between core network operations and base station operations may differ from that of the LTE or even be non-existent. The gNB is a next generation Node B (or, new Node B) supporting the <NUM> network (i.e., the NR).

Satellite communication may utilize geostationary earth orbit (GEO) satellite systems, but also low earth orbit (LEO) satellite systems, in particular mega-constellations (systems in which hundreds of (nano)satellites are deployed).

It is obvious for a person skilled in the art that the depicted system is only an example of a part of a radio access system and in practice, the system may comprise a plurality of (e/g)NodeBs, the user equipment may have an access to a plurality of radio cells and the system may comprise also other apparatuses, such as physical layer relay nodes or other network elements, etc. At least one of the (e/g)NodeBs or may be a Home(e/g)nodeB.

Embodiments may be implemented in software, hardware, application logic or a combination of software, hardware and application logic. In the context of this document, a "memory" or "computer-readable medium" may be any media or means that can contain, store, communicate, propagate or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer.

Reference to, where relevant, "computer-readable storage medium", "computer program product", "tangibly embodied computer program" etc., or a "processor" or "processing circuitry" etc. should be understood to encompass not only computers having differing architectures such as single/multi-processor architectures and sequencers/parallel architectures, but also specialized circuits such as field programmable gate arrays FPGA, application specify circuits ASIC, signal processing devices and other devices. References to computer readable program code means, computer program, computer instructions, computer code etc. should be understood to express software for a programmable processor firmware such as the programmable content of a hardware device as instructions for a processor or configured or configuration settings for a fixed function device, gate array, programmable logic device, etc..

Although the above examples describe embodiments of the invention operating within a wireless device or a gNB, it would be appreciated that the invention as described above may be implemented as a part of any apparatus comprising a circuitry in which radio frequency signals are transmitted and/or received. Thus, for example, embodiments of the invention may be implemented in a mobile phone, in a base station, in a computer such as a desktop computer or a tablet computer comprising radio frequency communication means (e.g. wireless local area network, cellular radio, etc.).

Embodiments of the inventions may be practiced in various components such as integrated circuit modules, field-programmable gate arrays (FPGA), application specific integrated circuits (ASIC), microcontrollers, microprocessors, a combination of such modules.

Claim 1:
A method comprising
obtaining, in an encoder, frames of video data for transmission;
providing the frames of the video data for rate control, wherein the rate control comprises:
obtaining a target value for at least one of an opinion score and a per-user opinion score;
obtaining a normalization quantization parameter;
using the normalization quantization parameter to encode and decode a particular frame of the video data,
calculating an objective distortion metric associated to the normalization quantization parameter for the particular frame;
executing a convolutional neural network to extract at least one feature indicative of at least one of perceptual quality and complexity of the particular frame in view of compression efficiency;
using a dense neural network and a regressor to determine a value for a target quantization parameter to achieve the target value when compressing the particular frame using the target quantization parameter, wherein the dense neural network has been previously trained using, as inputs, (i) extracted features of training frames of video data, (ii) normalization quantization parameters having been used to encode said training frames, with their associated objective distortion metrics, and (iii) target quantization parameters that relate to target opinion scores expected after encoding said training frames;
wherein the dense neural network uses the extracted features, the normalization quantization parameter and the objective distortion metric associated to the particular frame as inputs to determine the target quantization parameter; and
wherein the method further comprises encoding the particular frame with the target quantization parameter to generate an encoded frame.