Package: ray-for-mars
Ray provides a simple, universal API for building distributed applications.Ray is packaged with the following libraries for accelerating machine learning workloads:Tune: Scalable Hyperparameter TuningRLlib: Scalable Reinforcement LearningTrain: Distributed Deep Learning (beta)Datasets: Distributed Data Loading and ComputeAs well as libraries for taking ML and distributed apps to production:Serve: Scalable and Programmable ServingWorkflows: Fast, Durable Application Flows (alpha)There are also manycommunity integrationswith Ray, includingDask,MARS,Modin,Horovod,Hugging Face,Scikit-learn, and others. Check out thefull list of Ray distributed libraries here.Install Ray with:pip install ray. For nightly wheels, see theInstallation page.Quick StartExecute Python functions in parallel.importrayray.init()@ray.remotedeff(x):returnx*xfutures=[f.remote(i)foriinrange(4)]print(ray.get(futures))To use Ray’s actor model:importrayray.init()@ray.remoteclassCounter(object):def__init__(self):self.n=0defincrement(self):self.n+=1defread(self):returnself.ncounters=[Counter.remote()foriinrange(4)][c.increment.remote()forcincounters]futures=[c.read.remote()forcincounters]print(ray.get(futures))Ray programs can run on a single machine, and can also seamlessly scale to large clusters. To execute the above Ray script in the cloud, just downloadthis configuration file, and run:ray submit [CLUSTER.YAML] example.py--startRead more aboutlaunching clusters.Tune Quick StartTuneis a library for hyperparameter tuning at any scale.Launch a multi-node distributed hyperparameter sweep in less than 10 lines of code.Supports any deep learning framework, including PyTorch,PyTorch Lightning, TensorFlow, and Keras.Visualize results withTensorBoard.Choose among scalable SOTA algorithms such asPopulation Based Training (PBT),Vizier’s Median Stopping Rule,HyperBand/ASHA.Tune integrates with many optimization libraries such asFacebook Ax,HyperOpt, andBayesian Optimizationand enables you to scale them transparently.To run this example, you will need to install the following:$pipinstall"ray[tune]"This example runs a parallel grid search to optimize an example objective function.fromrayimporttunedefobjective(step,alpha,beta):return(0.1+alpha*step/100)**(-1)+beta*0.1deftraining_function(config):# Hyperparametersalpha,beta=config["alpha"],config["beta"]forstepinrange(10):# Iterative training function - can be any arbitrary training procedure.intermediate_score=objective(step,alpha,beta)# Feed the score back back to Tune.tune.report(mean_loss=intermediate_score)analysis=tune.run(training_function,config={"alpha":tune.grid_search([0.001,0.01,0.1]),"beta":tune.choice([1,2,3])})print("Best config: ",analysis.get_best_config(metric="mean_loss",mode="min"))# Get a dataframe for analyzing trial results.df=analysis.results_dfIf TensorBoard is installed, automatically visualize all trial results:tensorboard--logdir~/ray_resultsRLlib Quick StartRLlibis an industry-grade library for reinforcement learning (RL), built on top of Ray.
It offers high scalability and unified APIs for avariety of industry- and research applications.$pipinstall"ray[rllib]"tensorflow# or torchimportgymfromray.rllib.agents.ppoimportPPOTrainer# Define your problem using python and openAI's gym API:classSimpleCorridor(gym.Env):"""Corridor in which an agent must learn to move right to reach the exit.

    ---------------------
    | S | 1 | 2 | 3 | G |   S=start; G=goal; corridor_length=5
    ---------------------

    Possible actions to chose from are: 0=left; 1=right
    Observations are floats indicating the current field index, e.g. 0.0 for
    starting position, 1.0 for the field next to the starting position, etc..
    Rewards are -0.1 for all steps, except when reaching the goal (+1.0).
    """def__init__(self,config):self.end_pos=config["corridor_length"]self.cur_pos=0self.action_space=gym.spaces.Discrete(2)# left and rightself.observation_space=gym.spaces.Box(0.0,self.end_pos,shape=(1,))defreset(self):"""Resets the episode and returns the initial observation of the new one.
        """self.cur_pos=0# Return initial observation.return[self.cur_pos]defstep(self,action):"""Takes a single step in the episode given `action`

        Returns:
            New observation, reward, done-flag, info-dict (empty).
        """# Walk left.ifaction==0andself.cur_pos>0:self.cur_pos-=1# Walk right.elifaction==1:self.cur_pos+=1# Set `done` flag when end of corridor (goal) reached.done=self.cur_pos>=self.end_pos# +1 when goal reached, otherwise -1.reward=1.0ifdoneelse-0.1return[self.cur_pos],reward,done,{}# Create an RLlib Trainer instance.trainer=PPOTrainer(config={# Env class to use (here: our gym.Env sub-class from above)."env":SimpleCorridor,# Config dict to be passed to our custom env's constructor."env_config":{# Use corridor with 20 fields (including S and G)."corridor_length":20},# Parallelize environment rollouts."num_workers":3,})# Train for n iterations and report results (mean episode rewards).# Since we have to move at least 19 times in the env to reach the goal and# each move gives us -0.1 reward (except the last move at the end: +1.0),# we can expect to reach an optimal episode reward of -0.1*18 + 1.0 = -0.8foriinrange(5):results=trainer.train()print(f"Iter:{i}; avg. reward={results['episode_reward_mean']}")After training, you may want to perform action computations (inference) in your environment.
Here is a minimal example on how to do this. Alsocheck out our more detailed examples here(in particular fornormal models,LSTMs,
andattention nets).# Perform inference (action computations) based on given env observations.# Note that we are using a slightly different env here (len 10 instead of 20),# however, this should still work as the agent has (hopefully) learned# to "just always walk right!"env=SimpleCorridor({"corridor_length":10})# Get the initial observation (should be: [0.0] for the starting position).obs=env.reset()done=Falsetotal_reward=0.0# Play one episode.whilenotdone:# Compute a single action, given the current observation# from the environment.action=trainer.compute_single_action(obs)# Apply the computed action in the environment.obs,reward,done,info=env.step(action)# Sum up rewards for reporting purposes.total_reward+=reward# Report results.print(f"Played 1 episode; total-reward={total_reward}")Ray Serve Quick StartRay Serveis a scalable model-serving library built on Ray. It is:Framework Agnostic: Use the same toolkit to serve everything from deep
learning models built with frameworks like PyTorch or Tensorflow & Keras
to Scikit-Learn models or arbitrary business logic.Python First: Configure your model serving declaratively in pure Python,
without needing YAMLs or JSON configs.Performance Oriented: Turn on batching, pipelining, and GPU acceleration to
increase the throughput of your model.Composition Native: Allow you to create “model pipelines” by composing multiple
models together to drive a single prediction.Horizontally Scalable: Serve can linearly scale as you add more machines. Enable
your ML-powered service to handle growing traffic.To run this example, you will need to install the following:$pipinstallscikit-learn$pipinstall"ray[serve]"This example runs serves a scikit-learn gradient boosting classifier.importpickleimportrequestsfromsklearn.datasetsimportload_irisfromsklearn.ensembleimportGradientBoostingClassifierfromrayimportserveserve.start()# Train model.iris_dataset=load_iris()model=GradientBoostingClassifier()model.fit(iris_dataset["data"],iris_dataset["target"])@serve.deployment(route_prefix="/iris")classBoostingModel:def__init__(self,model):self.model=modelself.label_list=iris_dataset["target_names"].tolist()asyncdef__call__(self,request):payload=awaitrequest.json()["vector"]print(f"Received flask request with data{payload}")prediction=self.model.predict([payload])[0]human_name=self.label_list[prediction]return{"result":human_name}# Deploy model.BoostingModel.deploy(model)# Query it!sample_request_input={"vector":[1.2,1.0,1.1,0.9]}response=requests.get("http://localhost:8000/iris",json=sample_request_input)print(response.text)# Result:# {#  "result": "versicolor"# }More InformationDocumentationTutorialBlogRay 1.0 Architecture whitepaper(new)Exoshuffle: large-scale data shuffle in Ray(new)RLlib paperRLlib flow paperTune paperOlder documents:Ray paperRay HotOS paperGetting InvolvedForum: For discussions about development, questions about usage, and feature requests.GitHub Issues: For reporting bugs.Twitter: Follow updates on Twitter.Slack: Join our Slack channel.Meetup Group: Join our meetup group.StackOverflow: For questions about how to use Ray.