A distributed system for executing machine learning, comprising: a parameter server configured to maintain parameters of a plurality of machine learning models, wherein the parameters have a form of a key-value pair, each of the machine learning model corresponds to a set of key-value pairs in which keys of different key-value pairs are different; and when the parameter server is saving the parameters of the plurality of machine learning models, the parameter server saves a plurality of key-value pairs having a same key in a form in which a single key corresponds to a plurality of values, so as to merge and save the plurality of key-value pairs corresponding to different machine learning models with the same key as a form in which a single key corresponds to a plurality of values; and a plurality of computing devices (2000-1, 2000-2,..., 2000-n) configured to execute an algorithm regarding the plurality of machine learning models in parallel, wherein the algorithm is a training algorithm regarding the machine learning model or a prediction algorithm regarding the machine learning model; wherein the parameter server provides the computing device with the parameters that the computing device needs to read in order to execute the training algorithm regarding the machine learning model, so as for the computing device to execute the training algorithm using the read parameters, and the parameter server updates the parameters of the machine learning model according to a result of executing the training algorithm by the computing device; and/or, the parameter server provides the computing device with the parameters that the computing device needs to read in order to execute the prediction algorithm regarding the machine learning model, so as for the computing device to execute the prediction algorithm using the read parameters. a parameter server configured to maintain parameters of a plurality of machine learning models, wherein the parameters have a form of a key-value pair, each of the machine learning model corresponds to a set of key-value pairs in which keys of different key-value pairs are different; and when the parameter server is saving the parameters of the plurality of machine learning models, the parameter server saves a plurality of key-value pairs having a same key in a form in which a single key corresponds to a plurality of values, so as to merge and save the plurality of key-value pairs corresponding to different machine learning models with the same key as a form in which a single key corresponds to a plurality of values; and a plurality of computing devices (2000-1, 2000-2,..., 2000-n) configured to execute an algorithm regarding the plurality of machine learning models in parallel, wherein the algorithm is a training algorithm regarding the machine learning model or a prediction algorithm regarding the machine learning model; wherein the parameter server provides the computing device with the parameters that the computing device needs to read in order to execute the training algorithm regarding the machine learning model, so as for the computing device to execute the training algorithm using the read parameters, and the parameter server updates the parameters of the machine learning model according to a result of executing the training algorithm by the computing device; and/or, the parameter server provides the computing device with the parameters that the computing device needs to read in order to execute the prediction algorithm regarding the machine learning model, so as for the computing device to execute the prediction algorithm using the read parameters.