A computer system for distributed training of a machine learning model, the computer system comprising: a bulk synchronous parallel, BSP, system including a central BSP control module and at least two local BSP modules; at least two machine learning modules, each associated with exactly one of the at least two local BSP modules; and a shared memory module associated with exactly one pair of a local BSP module and a machine learning module; wherein the central BSP control module is configured to instruct the at least two local BSP modules to store, in their associated shared memory module, a local model; wherein each of the at least two machine learning modules is configured to read, from its associated shared memory module, the local model, compute a gradient based on the local model, and aggregate the gradient after its computation into an aggregated gradient in its associated shared memory module; and wherein each of the at least two local BSP modules is configured to communicate with a parameter server, PS, in order to receive a PS model that is to be stored as the local model, wherein the central BSP control module is further configured to instruct the at least two local BSP modules to periodically read out its associated shared memory module, and, after each reading out the shared memory module of the periodically reading out the shared memory module, to provide, to the PS, the aggregated gradient for updating, in the PS, the PS model, wherein the PS model update is decoupled from the gradient computations; wherein the central BSP control module is configured to notify the at least two local BSP modules on the availability of an updated PS model residing in the PS, and each of the at least two local BSP modules is configured to download the updated PS model from the PS and to use it to update the local model stored in its associated shared memory module. a bulk synchronous parallel, BSP, system including a central BSP control module and at least two local BSP modules; at least two machine learning modules, each associated with exactly one of the at least two local BSP modules; and a shared memory module associated with exactly one pair of a local BSP module and a machine learning module; wherein the central BSP control module is configured to instruct the at least two local BSP modules to store, in their associated shared memory module, a local model; wherein each of the at least two machine learning modules is configured to read, from its associated shared memory module, the local model, compute a gradient based on the local model, and aggregate the gradient after its computation into an aggregated gradient in its associated shared memory module; and wherein each of the at least two local BSP modules is configured to communicate with a parameter server, PS, in order to receive a PS model that is to be stored as the local model, wherein the central BSP control module is further configured to instruct the at least two local BSP modules to periodically read out its associated shared memory module, and, after each reading out the shared memory module of the periodically reading out the shared memory module, to provide, to the PS, the aggregated gradient for updating, in the PS, the PS model, wherein the PS model update is decoupled from the gradient computations; wherein the central BSP control module is configured to notify the at least two local BSP modules on the availability of an updated PS model residing in the PS, and each of the at least two local BSP modules is configured to download the updated PS model from the PS and to use it to update the local model stored in its associated shared memory module.