Patent ID: 12198020

It should be noted that the figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

LIST OF REFERENCE NUMBERS

The following list of reference numbers is provided for facilitating the interpretation of the drawings and shall not be construed as limiting the scope of the present invention.20sensor40actuator60physical environment100system for training machine learnable model102system for using machine learned model for inference150system memory160processor subsystem180data storage interface190data storage192training data for training194input data for inference196data representation of machine learnable model198data representation of machine learned model200method of training machine learnable model202method of using machine learned model for inference210accessing training data for training212accessing input data for inference220training machine learnable model222using machine learned model for inference230providing state memory240extracting previous internal state information250updating state memory with current internal state300autonomous system310motor commands320physical environment330delayed measurements400binary memory frames410binary last memory450train temporal attention model500autonomous system510physical environment520predict current sensor measurement using binary last memory530predict future sensor measurement from current action540control of autonomous system based on sensor prediction550training loss600LSTM layer610hidden memory state620output state650LSTM layer using binary last memory660hidden memory state670output of binary last memory680temporal attention model685float to binary conversion690binary last memory temporal update700system for control or monitoring using machine learned model720sensor data interface722sensor data740actuator interface742control data750system memory760processor subsystem780data storage interface790data storage800physical environment810(semi-)autonomous vehicle820camera sensor830electric motor900computer-readable medium910non-transitory data

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

The following provides with reference toFIG.1a schematic overview of a system for training a machine learnable model, which also is a schematic representation of a system for using a machine learned model for inference, and with reference toFIG.2a schematic overview of a computer-implemented method for training a machine learnable model, which also is a schematic representation of a computer-implemented method for using a machine learned model for inference. Each described system and method uses a state memory which efficiently stores previous states of the machine learn(ed)(able) model during training and inference. The state memory itself is further explained with reference toFIGS.3-9.FIGS.10and11further relate to the use of the machine learned model for the control or monitoring of a physical system, such as an (semi-)autonomous vehicle, whileFIG.12relates to a computer-readable medium comprising a computer program.

FIG.1shows a schematic representation of a system100for training a machine learnable model, which is also a schematic representation of a system102for using a machine learned model for inference. The system100,102may comprise an input interface for accessing input data192,194for the machine learnable model, being either training data192or non-training type input data194, such as sensor data. For example, as illustrated inFIG.1, the input interface may be constituted by a data storage interface180which may access the respective data192,194from a data storage190. For example, the data storage interface180may be a memory interface or a persistent storage interface, e.g., a hard disk or an SSD interface, but also a personal, local or wide area network interface such as a Bluetooth, Zigbee or Wi-Fi interface or an ethernet or fiberoptic interface. The data storage190may be an internal data storage of the system100,102, such as a hard drive or SSD, but also an external data storage, e.g., a network-accessible data storage.

In embodiments of the training system100, the data storage190may further comprise a data representation196of an untrained version of the machine learnable model which may be accessed by the system100from the data storage190. It will be appreciated, however, that the training data192and the data representation196of the untrained machine learnable model may also each be accessed from a different data storage, e.g., via a different subsystem of the data storage interface180. Each subsystem may be of a type as is described above for the data storage interface180. In other embodiments, the data representation196of the untrained machine learnable model may be internally generated by the system100on the basis of design parameters for the machine learnable model, and therefore may not explicitly be stored on the data storage190or elsewhere.

In embodiments of the system102which uses the machine learned model for inference, the data storage190may further comprise a data representation198of a trained version of the machine learnable model, which is here and elsewhere also referred to as a machine learned model, and which may be accessed by the system100from the data storage190. It will be appreciated, however, that the input data194and the data representation198of the machine learned model may also each be accessed from a different data storage, e.g., via a different subsystem of the data storage interface180. Each subsystem may be of a type as is described above for the data storage interface180. In some embodiments, the input interface for accessing the input data194may be a sensor interface for accessing sensor data, as also further described with reference toFIG.10.

The system100,102is further shown to comprise a system memory150, which may for example be a random access-based system memory or in general any suitable type of system memory for storing and randomly accessing a data structure representing a state memory of a type as described elsewhere in this specification.

The training system100may further comprise a processor subsystem160which may be configured to, during operation of the system100, provide the state memory, e.g., by allocating and providing a corresponding data structure in the system memory150. The state memory may comprise, for each element of the internal state, a value X(t) which is indicative of a most recent occurrence of the element holding or transitioning to a particular binary state value, wherein the most recent occurrence is expressed as a number of training steps relative to the current training step. The processor subsystem160may be further configured to, during the training, in a current training step, extracting, from the state memory, previous internal state information for use in determining a current internal state of the machine learnable model, and after determining the current internal state of the machine learnable model, updating the state memory with the current internal state by, for each element of the internal state, updating the corresponding value of the state memory.

With continued reference toFIG.1, the system102for using the machine learned model for inference may further comprise a processor subsystem160which may be configured to, during operation of the system102, provide a state memory, e.g., by allocating and providing a corresponding data structure in the system memory150. The state memory may comprise, for each element of the internal state, a value X(t) which is indicative of a most recent occurrence of the element holding or transitioning to a particular binary state value, wherein the most recent occurrence is expressed as a number of inference steps relative to the current inference step. The processor subsystem160may be further configured to, during the inference, in a current inference step, extracting, from the state memory, previous internal state information for use in determining a current internal state of the machine learned model, and after determining the current internal state of the machine learned model, updating the state memory with the current internal state by, for each element of the internal state, updating the corresponding value of the state memory.

The operation of the state memory and its role in the training of and inference by the machine learnable model will be further described with reference toFIGS.3-9.

The training system100may further comprise an output interface for outputting a data representation198of the machine learned model, this data also being referred to as trained model data198. For example, as also illustrated inFIG.1, the output interface may be constituted by the data storage interface180, with said interface being in these embodiments an input/output (‘IO’) interface, via which the trained model data198may be stored in the data storage190. For example, the data representation196defining the ‘untrained’ machine learnable model may during or after the training be replaced, at least in part, by the data representation198of the machine learned model, in that the parameters of the machine learnable model, such as weights, hyperparameters and other types of parameters of machine learnable models, may be adapted to reflect the training on the training data192. In other embodiments, the data representation198may be stored separately from the data representation196of the ‘untrained’ machine learnable model. In some embodiments, the output interface may be separate from the data storage interface180, but may in general be of a type as described above for the data storage interface180.

FIG.2shows a schematic representation of a computer-implemented method200for training a machine learnable model, which is also a schematic representation of a computer-implemented method202for using a machine learned model for inference.

The method200is shown to comprise, in a step titled “ACCESSING TRAINING DATA FOR TRAINING”, accessing210training data for the machine learnable model, wherein the training data comprises a temporal sequence of input data instances. The method200is further shown to comprise, in a step titled “TRAINING MACHINE LEARNABLE MODEL”, training220the machine learnable model in a series of trainings steps on respective input data instances, wherein, in a respective training step, the machine learnable model assumes an internal state which is comprised of or representable as a set of binary values representing respective elements of the internal state, wherein the internal state depends on one or more previous internal states of the machine learnable model. The method200is further shown to comprise, in a step titled “PROVIDING STATE MEMORY”, providing230a state memory which comprises, for each element of the internal state, a value which is indicative of a most recent occurrence of the element holding or transitioning to a particular binary state value, wherein the most recent occurrence is expressed as a number of training steps relative to the current training step. The method200is further shown to comprise, during the training and in a current training step thereof, in a step titled “EXTRACTING PREVIOUS INTERNAL STATE INFORMATION”, extracting240, from the state memory, previous internal state information for use in determining a current internal state of the machine learnable model, and in a step titled “UPDATING STATE MEMORY WITH CURRENT INTERNAL STATE” and after determining the current internal state of the machine learnable model, updating250the state memory with the current internal state by, for each element of the internal state, updating the corresponding value of the state memory.

The method202corresponds to the method200in as far as described with reference toFIG.2, except that the method uses the machine learned model in step222for inference instead of training the machine learnable model and that the accessing step210is replaced by a step titled “ACCESSING INPUT DATA FOR INFERENCE” which comprises accessing212input data for the machine learned model, wherein the input data comprises a temporal sequence of input data instances. Otherwise, the above description ofFIG.2applies, except for the training and training steps now being inference and inference steps.

The following examples describe the state memory and its use in more detail. However, the actual implementation of the state memory and its use may be carried out in various other ways, e.g., on the basis of analogous mathematical concepts. Various other embodiments are within reach of the skilled person based on this specification.

Binary (sparse) representations [1] may be used to encode information. For binary representations, the activation xi(t) of a single unit of a machine learnable model (e.g., a neuron) may only take the values 0 or 1. The following only considers such binary states x(t) of a machine learnable model. However, intermediate computation results may not need to be binary, but for example floating-point numbers. For example, in case of a linear mapping y=W*x(t), where W is a parameter matrix, the input x(t) may be binary, but the (intermediate) result y may be a floating-point vector which may, for example, rounded again to a binary representation of the machine learnable model's current state:
yi≥0→xi(t+1):=1
yi<0→xi(t+1):=0

FIG.3illustrates a delayed transformation problem, illustrating one of the relevant application areas for the state memory as described in this specification. This problem relates to following: a challenge for autonomous systems may be to estimate and model temporal delays. For example, a current action u(t)310, e.g., as affected by an actuator of the autonomous system300, may casually affect the sensor measurements s(t)330of the autonomous system with some time delay d. The sensor measurement may be partially affected by the system's actions u(t) through complex real-world physics F of the physical environment320in which the current action u(t) is performed:
s(t)=F(u(t−d))

Here, F may be an unknown and complex function of the physical environment320. InFIG.3, this time delay d is illustrated by two graphs, representing the current action u(t) and the sensor measurement s(t). Here, the time axis t runs from past (left) to future (right). For simplicity, is assumed that the delay d is constant and independent from the action u, which sometimes is the case but not necessarily has to be the case.

A goal may be to have a machine learnable model learn to predict the future sensor measurement while F and d are both unknown. For this purpose, recurrent and similar types of machine learnable models may be used, in which a current state depends on past state information, thereby effectively providing the machine learnable model with a memory. To be able to cope with a delay d, the memory of the machine learnable model may need to have an appropriate memory depth. Namely, the past states of the machine learnable model need to be stored, both during training and inference, for as long as is necessary to account for the delay d, e.g., for at least d timesteps of the training/inference if d is expressed correspondingly. The state memory as described in the following and elsewhere in this specification provides an efficient way of storing past state information.

FIG.4shows an example in which a 4-dimensional binary sequence of states x=(x1, x2, x3, x4), e.g., a four-dimensional state vector, is to be stored. In this example, on the left-hand side, the storage of individual binary frames is shown, which is here and elsewhere also referred to as a ‘binary dense memory’400(BDM). On the right-hand side, the storage of the past state information is shown using the state memory, also referred to as ‘binary last memory’410(BLM). In this example, 4 bits (b=4) are used for each memory. It can be seen that in the case of the binary dense memory400, this means that four binary frames may be stored, i.e., the last four binary states. The binary last memory410may replace the b bit dense memory, which encodes the last b=4 frames, with another memory structure which also uses b bits but instead of encoding all b previous states, the b bits may be used in this example to store a single (integer) duration until the last onset, which is in this and some other examples the last time of a unit holding the binary value ‘1’, e.g., xi(t)=1. As such, this last onset is the last moment in time at which the respective unit assumed the particular binary state value ‘1’. This state may represent an event, e.g., as detected by an event-based sensor, and which resulted in an activation of a particular unit of the machine learnable model, e.g., causing the state to assume the binary state value ‘1’.

Hence, a single b bit integer unit (e.g., b=4) may be used for every binary state element/unit of the machine learnable model to encode the (frame) duration since this units last onset. For example, if the memory of a unit is Xi(t)=0 at time frame t, this may mean that this unit is active at t: xi(t)=0, while a memory of Xi(t)=5 may mean that xiwas activated 5 time frames ago: xi(t−5)=xi(t−Xi(t))=1 but was inactive in the frames in between: xi(t−5: t)=(1, 0, 0, 0, 0, 0) and hence that Xi(t−1)=4, Xi(t−2)=3, Xi(t−3)=2, Xi(t−4)=1, Xi(t−5)=0. As soon as xiis activated again at some future time frame tnew, the corresponding value stored in the state memory may be overwritten Xi(tnew)=0 and the duration for this new onset may be stored in the state memory. More generally, this memory X, of the time duration since the last event may be updated as:
xi(t)=1→Xi(t):=0
xi(t)=0→Xi(t):=min{2b−1,Xi(t−1)+1}

Note that for the BDM400, the used bits b directly yield the time horizon of the dense memory T=b, e.g., T=4 for theFIG.4example, while for the BLM410, the maximally covered delay is T=2b−1 which is exponentially larger than b, for growing b.

Similar to the BDM, the BLM410encodes not only the samples themselves, but also their temporal order, e.g. which states temporally precede which other states. For the BLM410, the order may be encoded transparently via the different delays of the states.

FIG.5relates to so-called ‘temporal aware computations’ in which past state information may again be extracted from the state memory, e.g., the BLM, using a temporal attention model (TMP_ATT) which may be trainable and which may generate one or more internal states for the machine learnable model. The one or more generated internal states thus represent previous internal state information, which may be selected as being relevant in accordance with the temporal attention model, and may be used by the machine learnable model to determine its current internal state.FIG.5shows an example of a trainable temporal attention model TMP_ATT which has learned to extract the binary state x(t−3) from 3 frames before from the binary last memory. In general, this temporal attention model may accept parameters A and the values X stored in the BLM as input:
y=TMP_ATT(A,X)

The temporal attention model, which may in the following also be referred to as an attention mechanism or as an attention function, may function like an activation function which may be applied coefficient-wise on every X and may yield one or more binary states y which may have the same dimension as X. Broadly speaking, TMP_ATT may extract a binary state y from the memory X, with the extracted binary state y being the state some frames ago or a combination over several frames ago. The way TMP_ATT may select this stored information may be parameterized by A. Many different types of temporal attention models may be used, which may be categorized in at least two different classes:trainable: the parameters A may be trained, for example in a similar manner and in parallel to the machine learnable model, for example to its neural network parameters. The parameters A may then be constant after the training. For example, TMP_ATT may have learned to always extract the state for a certain delay d from X.not-trainable: the parameters A may not be trained and may either be constant or themselves produced during runtime, e.g., during training or interference, e.g., as activation of a layer in the network. A beneficial use of a not-trainable parametrization of TMP_ATT may be the following: the delay d for which the state may be extracted from the BLM X may vary during runtime and may for example be dependent on some other network state A=A(x(t)). For the BLM, the temporal attention mechanism TMP_ATT may be used to extract information from X and generate a new binary state y which may then, for example, be used as input to a parameterized network layer, e.g., W*TMP_ATT(A,X(t)).

FIG.5shows an example of a trainable TMP_ATT which has learned, by way of training450, to extract a delayed state, namely with d=3, by learning to parametrize A as a piece-wise step function over potential delays. Shown in the left-hand upper side are initial values for the individual parameters of A=(A0, A1, A2, A3, A4), which after the training450have resulted in A=(0.0, 0.0, 0.0, 1.0, 0.0), indicating that the temporal attention model has learned to extract a delayed state with d=3. This extraction is illustrated in the left-hand lower side, illustrating both the contents of the BLM410and the corresponding BDM400. Here, binary state values of ‘1’ are shown as ‘x’. It can be seen that the temporal attention model TMP_ATT has learned to provide as output the delayed state of 3 frames ago.

Another beneficial use of a temporal attention model is that of temporal scaling of the input BLM X. In a simple example, TMP_ATT may first temporally normalize the input BLM X, e.g. using its mean or maximum duration:

X=Xmean⁡(X)⁢⁢or⁢⁢X=Xmax⁡(X)

before extracting stored information for certain (then normalized) delays. This normalization may introduce a temporal invariance of the generated output state with respect to the input BLM X. As such, for certain temporal scaling, e.g., using the mean or the maximum, the relative temporal durations of states in the sequence may stay constant.

It is noted that instead of using a temporal attention model to extract previous internal state information from the state memory, such previous internal state information may also be extracted in any other way, e.g., by manually designed heuristics, such as a rule-based system.

For example, such heuristics may be designed to reconstruct one or more previous internal states of the machine learnable model from the values in the state memory representing the respective most recent occurrences. Such a reconstruction may for example reconstruct four binary state frames from X(t=0)=(2, 0, 5, 15), for example, by assuming that the activation represented by the stored value is the only activation within the reconstructed time window, e.g., that the binary state values preceding a last activation are ‘0’. In many cases, the reconstruction error may be small, e.g., if events are incidental seldom and only short-lived, and/or the binary state values preceding a last activation may be of lesser relevance to the training of or inference by the machine learnable model.

FIG.6shows an example of a binary sequence containing states x(t) of dimension8and in which a binary last memory410of 8 bits is used (the state dimension and BLM bits are only equal by coincidence), which is shown next to a binary dense memory400of 136 bits. In this example, the BLM410can store delays between 0 and 255 and thereby capture the delays of 43 and 88 which are needed to represent the binary sequence ofFIG.6. It can be seen that the 8-bit BLM410can reliably store the binary sequence. A corresponding BDM would have needed at least 88 bits for capturing the binary sequence.

FIG.7illustrates an application of estimating a delayed transformation, e.g., as previously described with reference toFIG.3, and using a learned past-to-present prediction to generate a present-to-future prediction in order to address a predictive control task, such as control of an autonomous system. In general, in cases of a delay transformation, the temporal delay d may either be constant and independent from time and also its estimate destimmay independent from time, or d may depend on some environmental state (e.g., the actual action of the system or some other environmental state) and hence, also the estimate may vary as a function of time: destim(t)=destim(X(t)).

The first variant of temporally constant delay is a sub-case of the more general temporally dependent variant but may be implemented in a more efficient way. Namely, in case of a temporally constant delay d, the delay estimate may be implemented as a trainable parameter of the machine learnable model and may stay constant after training and during inference. In case of a varying, time dependent delay d(t), the machine learnable model may compute an estimate of the delay destim(t) at every time step.

FIG.7gives a schematic overview for the case of a trainable and constant destim. As in the case ofFIG.3, this application concerns an autonomous system500which performs actions u(t) in a physical environment510, e.g., via an actuator, and obtains sensor measurements s(t). In this example, a machine learnable model is learned to predict520the current sensor measurement s(t) by spred(t) using a past state u(t−d) (in particular its BLM-stored representation U(t)) and a training loss550. This past-to-present function, as learned by the machine learnable model, may be used to predict530a future sensor state spred(t+d_estim) from a present control action u(t). The anticipated sensor signal may now be used for predictive control540of the actuator or for similar purposes.

As also indicated elsewhere, since the BLM stores past state information, it is particularly useful in applications where temporal information, such as sensor data, is processed and decision making and/or control is based on temporal information. For such applications, often recurrent machine learnable models, such as neural networks, are used.

FIG.8shows a standard LSTM layer600which is frequently used in recurrent neural networks. The LSTM layer600is shown to comprise a hidden memory state610and an output state H620, representing an output of a previous application of the LSTM layer600and thus essentially a previous state which would conventionally be stored integrally.

FIG.9shows a modified LSTM layer650which incorporates a binary last memory. Like the standard LSTM layer600, the modified LSTM layer650comprises a hidden memory state660. However, the previous output state H620which is used as input to the LSTM layer650is now replaced by an output670of a binary last memory. In addition, a temporal attention model T680is used to extract past state information from the output of the binary last memory. The temporal attention model T680may be a parameterized TMP_ATT( ) function extracting or learning to extract parts from the binary last memory. The current state of the modified LSTM layer650, which may comprise floating point values, may be converted via a float-to-binary conversion B685. For example, the conversion B may comprise applying an element-wise step function to the floating-point values or a Bernoulli distribution parameterized by the output of the output gate. Finally, the resulting binary state values may be used to update690the binary last memory, e.g., in a manner as described elsewhere in this specification. For applications where standard LSTM are deployed, the BLM LSTM variant may be used to improve performance, e.g., due to temporal memory of layer input X and output H, and reduced memory footprint due to only binary input/output.

It is noted that while some of the examples described in this specification relate to the storing, in the state memory, of the most recent occurrence of a state element having the binary state value ‘1’, the state memory may also store the most recent occurrence of the state element having the binary state value ‘0’. Alternatively, the state memory may store a most recent occurrence of a state element transitioning to a particular binary state value, thereby encoding the last change to a particular binary state value, or in general the most recent occurrence of any transition to another binary state value. Such type of memory may also be relevant for some applications. Another example is that the state memory comprises, for each element of the internal state, a further value. The state memory may then be updated at each step so that the (first) value is indicative of the most recent occurrence at which the element transitioned to a first binary state value, such as ‘0’, and the further value is indicative of the most recent occurrence at which the element transitioned to a second binary state value, e.g., ‘1’, which is different from the first binary state value. Accordingly, the state memory may store the last occurrence of each type of transition.

FIG.10shows a system700for controlling or monitoring a physical system using a machine learned model. Such a system700may represent a specific example of a system configured to use the machine learned model for inference purposes. The system700may comprise an input interface780for accessing a data representation198of the machine learned model as may be generated by the system100ofFIG.1or the method200ofFIG.2or as described elsewhere. For example, as also illustrated inFIG.10, the input interface may be constituted by a data storage interface780which may access the machine learned model198from a data storage790. In general, the input interface780and the data storage790may be of a same type as described with reference toFIG.1for the input interface180and the data storage190.FIG.10further shows the data storage792comprising input data722to be used as input to the machine learned model to obtain output data representing an inference of the machine learned model. For example, the input data722may be or may comprise sensor data obtained from one or more sensors. A specific example, the input data722may represent an output of a sensor-based observation of a current state of the physical system, e.g., a sensor measurement, and the machine learned model may provide an inference based on the current state of the physical system, which may in a specific example be an inference relating to a future state of the physical system. In some embodiments, the sensor data as input data722may also be received directly from a sensor20, for example via a sensor interface720of via another type of interface instead of being accessed from the data storage790via the data storage interface780.

The system700may further comprise a processor subsystem760which may be configured to, during operation of the system700, apply the machine learned model to the input data722to obtain output data representing an inference by the machine learned model, wherein said applying may comprise providing and using a state memory as described elsewhere in this specification. The state memory may be allocated as a data structure in the system memory750. The obtained output data may take various forms, and may in some examples be a direct output of the system700. In other examples, which are also described in the following, the system700may output data which is derived from the inference of the machine learned model, instead of directly representing the inference.

It will be appreciated that the same considerations and implementation options apply for the processor subsystem760as for the processor subsystem160ofFIG.1. It will be further appreciated that the same considerations and implementation options may in general apply to the system700as for the system100ofFIG.1, unless otherwise noted.

FIG.10further shows various optional components of the system700. For example, in some embodiments, the system700may comprise a sensor data interface720for directly accessing sensor data722acquired by a sensor20in an environment60. The sensor20may but does not need to be part of the system700. The sensor20may have any suitable form, such as an image sensor, a lidar sensor, a radar sensor, a pressure sensor, a contain temperature sensor, etc. In some embodiments, the sensor data722may sensor measurements of different physical quantities in that it may be obtained from two or more different sensors sensing different physical quantities. The sensor data interface720may have any suitable form corresponding in type to the type of sensor, including but not limited to a low-level communication interface, e.g., based on I2C or SPI data communication, or a data storage interface of a type as described above for the data storage interface780.

In some embodiments, the system700may comprise an actuator interface740for providing control data742to an actuator40in the environment60. Such control data742may be generated by the processor subsystem760to control the actuator40based on one or more inferences, as may be generated by the machine learned model when applied to the input data722. For example, the actuator40may be an electric, hydraulic, pneumatic, thermal, magnetic and/or mechanical actuator. Specific yet non-limiting examples include electrical motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servomechanisms, solenoids, stepper motors, etc. Such type of control is described with reference toFIG.11for an (semi-)autonomous vehicle.

In other embodiments (not shown inFIG.10), the system700may comprise an output interface to a rendering device, such as a display, a light source, a loudspeaker, a vibration motor, etc., which may be used to generate a sensory perceptible output signal which may be generated based on one or more inferences by the machine learned model. The sensory perceptible output signal may be directly indicative of the inferences by the machine learned model, but may also represent a derived sensory perceptible output signal, e.g., for use in guidance, navigation or other type of control of the physical system.

In general, each system described in this specification, including but not limited to the systems100,102ofFIG.1and the system700ofFIG.10, may be embodied as, or in, a single device or apparatus, such as a workstation or a server. The device may be an embedded device. The device or apparatus may comprise one or more microprocessors which execute appropriate software. For example, the processor subsystem of the respective system may be embodied by a single Central Processing Unit (CPU), but also by a combination or system of such CPUs and/or other types of processing units. The software may have been downloaded and/or stored in a corresponding memory, e.g., a volatile memory such as RAM or a non-volatile memory such as Flash. Alternatively, the processor subsystem of the respective system may be implemented in the device or apparatus in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA). In general, each functional unit of the respective system may be implemented in the form of a circuit. The respective system may also be implemented in a distributed manner, e.g., involving different devices or apparatuses, such as distributed local or cloud-based servers. In some embodiments, the system102,700may be part of vehicle, robot or similar physical entity, and/or may be represent a control system configured to control the physical entity.

FIG.11shows an example of the above, in that the system700is shown to be a control system of an (semi-)autonomous vehicle810operating in an environment800. The autonomous vehicle810may be autonomous in that it may comprise an autonomous driving system or a driving assistant system, with the latter also being referred to as a semiautonomous system. The autonomous vehicle810may for example incorporate the system700to control the steering and the braking of the autonomous vehicle based on sensor data obtained from a camera sensor820integrated into the vehicle810. For example, the system700may control an electric motor820to perform (regenerative) braking in case the autonomous vehicle810is expected to collide with a traffic participant. The system700may control the steering and/or braking to avoid collision with the traffic participant. For that purpose, the system700may infer a current or future state of the vehicle with respect its environment, including the traffic participant, based on the sensor data obtained from the video camera. If the state of the vehicle, e.g., its position relative to the traffic participant, is expected to result in a collision, the system700may take corresponding action, for example by the aforementioned steering and/or braking.

Each method, algorithm or pseudo-code described in this specification may be implemented on a computer as a computer implemented method, as dedicated hardware, or as a combination of both. As also illustrated inFIG.12, instructions for the computer, e.g., executable code, may be stored on a computer-readable medium900, e.g., in the form of a series910of machine-readable physical marks and/or as a series of elements having different electrical, e.g., magnetic, or optical properties or values. The executable code may be stored in a transitory or non-transitory manner. Examples of computer-readable mediums include memory devices, optical storage devices, integrated circuits, servers, online software, etc.FIG.12shows an optical disc910. In an alternative embodiment of the computer-readable medium900, the computer-readable medium may comprise model data910defining a machine learned model as described elsewhere in this specification.

Examples, embodiments or optional features, whether indicated as non-limiting or not, are not to be understood as limiting the scope of the present invention.

It is noted that systems and computer-implemented methods are described for training a machine learnable model and for using the machine learned model for inference, both of which using only limited memory resources. During training and inference, the machine learnable model uses previous state information. A state memory is provided which efficiently stores this previous state information. Instead of storing each previous state individually and integrally, for each element of the internal state, a value is stored in the state memory which is indicative of a most recent occurrence of an element of the internal state of the machine learnable model holding or transitioning to a particular binary state value. This type of state memory has been found to be highly efficient for storing state information when the states of the machine learnable model are representable as binary values and when states infrequently hold or transition to a particular binary state value, e.g., if sensor events are infrequent. Due to the reduced memory footprint during training and inference, the applicability of the machine learnable model to real-life problems is increased.

It should be noted that the above-mentioned embodiments illustrate rather than limit the present invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the present invention. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or stages other than those stated. The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. Expressions such as “at least one of” when preceding a list or group of elements represent a selection of all or of any subset of elements from the list or group. For example, the expression, “at least one of A, B, and C” should be understood as including only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The present invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually separately does not indicate that a combination of these measures cannot be used to advantage.