A method for training a plurality of machine learning models in a computer system, wherein each machine learning model comprises a set of parameters, the method comprising: receiving, by a job server (110; 500) in communication with a plurality of compute nodes (130; 600) over a network, a plurality of training jobs,, including a first training job and a second training job, for training the machine learning models; allocating, by the job server, each training job to a respective training group of one or more of said compute nodes (130;600) based on current requirements of the training jobs and current status of the compute nodes, including the job server determining which compute nodes are included in which training group; and executing, by the training groups, their allocated training jobs, said execution comprising updating values for the parameters of the machine learning models; the method further characterised by: in response to determining that currently available computing nodes are insufficient to meet the requirements of the second training job, dynamically reallocating, by the job server, compute nodes currently allocated to and executing the first training job from the first training job to the second training job while the first training job is executing, in such a way to still meet a minimum required by said the first training job, while freeing up resources to meet a required minimum for said second training job; and communicating, by at least one of the training groups that comprises two or more compute nodes, the updated values of the parameters between said two or more compute nodes of said at least one of the training groups; and said at least one of the training groups using the communicated updated values in furtherance of the respective training job. receiving, by a job server (110; 500) in communication with a plurality of compute nodes (130; 600) over a network, a plurality of training jobs,, including a first training job and a second training job, for training the machine learning models; allocating, by the job server, each training job to a respective training group of one or more of said compute nodes (130;600) based on current requirements of the training jobs and current status of the compute nodes, including the job server determining which compute nodes are included in which training group; and executing, by the training groups, their allocated training jobs, said execution comprising updating values for the parameters of the machine learning models; the method further characterised by: in response to determining that currently available computing nodes are insufficient to meet the requirements of the second training job, dynamically reallocating, by the job server, compute nodes currently allocated to and executing the first training job from the first training job to the second training job while the first training job is executing, in such a way to still meet a minimum required by said the first training job, while freeing up resources to meet a required minimum for said second training job; and communicating, by at least one of the training groups that comprises two or more compute nodes, the updated values of the parameters between said two or more compute nodes of said at least one of the training groups; and said at least one of the training groups using the communicated updated values in furtherance of the respective training job. in response to determining that currently available computing nodes are insufficient to meet the requirements of the second training job, dynamically reallocating, by the job server, compute nodes currently allocated to and executing the first training job from the first training job to the second training job while the first training job is executing, in such a way to still meet a minimum required by said the first training job, while freeing up resources to meet a required minimum for said second training job; and communicating, by at least one of the training groups that comprises two or more compute nodes, the updated values of the parameters between said two or more compute nodes of said at least one of the training groups; and said at least one of the training groups using the communicated updated values in furtherance of the respective training job.