Package: iPLAN
This repository is the codebase for our paper.iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement LearningThis repository was originallyforked fromhttps://github.com/oxwhirl/pymarlandhttps://github.com/carolinewang01/dm2.
The MAPPO baseline comes fromhttps://github.com/uoe-agents/epymarl.Table of ContentsDependenciesInstallationRunning iPLANAblation StudyIPPO,IPPO-BM,IPPO-GAT,iPLAN-Hard,iPLAN-FCBaselinesHelper FunctionsCompute Navigation MetricsGenerate AnimationPlot Reward CurveResultsAnimationCitationDependenciesPyTorch(1.13.1 + cu116) (GPU)stable-baselines3Heterogeneous_Highway_Env(Forked fromhighway-env)scaredPyYAMLNote: Please our modified Highway-env given inHeterogeneous_Highway_Envas there are major changes from the initial version ofhighway-env. Also,Multi-agent Particlesused in our repo are different. Please use
the code given inenvs/mpefolder.InstallationFirst, install dependenciespip install stable-baselines3[extra] pyyaml sacredThen install our forked version of Highway-envpip install Heterogeneous_Highway_EnvFinally, install iPLAN packagepip install iPLANRunning iPLANIn the configuration fileconfig/default.yaml, set up environments needed for your experiment:Set environmentenv:MPEfor Non-cooperative Navigation andhighwayfor Heterogeneous HighwaySet difficulty leveldifficulty:easyfor easy (Non-cooperative Navigation) or mild (Heterogeneous Highway) scenario.hardfor hard (Non-cooperative Navigation) or chaotic (Heterogeneous Highway) scenario.SetBehavior_enable: TrueSetGAT_enable: TrueandGAT_use_behavior: TrueSetsoft_update_enable: Trueandbehavior_fully_connected: FalseRunpython3 main.pyResults, including printed logs, saved models and tensorboard logger, are stored in the folderresultsAblation StudyWhen running experiments for ablation study, please only change the hyperparameters mentioned
in the configuration fileconfig/default.yamland keep those the same as they are in the iPLAN experiment.Running IPPOSetBehavior_enable: FalseSetGAT_enable: FalseandGAT_use_behavior: FalseRunpython3 main.pyRunning IPPO-BMSetBehavior_enable: TrueSetGAT_enable: FalseandGAT_use_behavior: FalseRunpython3 main.pyRunning IPPO-GATSetBehavior_enable: FalseSetGAT_enable: TrueandGAT_use_behavior: TrueRunpython3 main.pyRunning iPLAN-HardSetsoft_update_enable: FalseRunpython3 main.pyRunning iPLAN-FCSetbehavior_fully_connected: TrueRunpython3 main.pyBaselinesBaselines used in this paper could be found in thebaselinesfolder, where the organization of files is
similar to the main directory of iPLAN. Please change the environment setting inconfig/default.yamlbefore experiments. No extra changes need.QMIX: Runpython3 baselines/QMIX/main.pyMAPPO: Runpython3 baselines/MAPPO/main.pyHelper FunctionsNotebooks for helper function are given inhelperfolder.
Please follow the instructions below:Compute Navigation Metrics(Only for Heterogeneous Highway)
In the configuration fileconfig/default.yamlSetmetrics_enable: TrueSetnum_test_episodeslarger thanbatch_size_runRunpython3 main.pyThen you will get printed navigation metrics after the execution logs of each episode.To compute the navigation metrics, use the notebookhelper/RL_results_metrics.ipynbto compute averaged navigation
metrics from the printed log file (usually given inresults/sacred).Generate Animation(Only for Heterogeneous Highway)
In the configuration fileconfig/default.yamlSetanimation_enable: True(Recommended) Setmetrics_enable: True, Setnum_test_episodeslarger thanbatch_size_runRunmain.pyScreenshots of the Heterogeneous Highway are stored in theanimationfolder. Use the notebookhelper/Gif_helper.ipynbto generate animation from screenshots.Plot Reward CurveThe printed log file are usually given inresults/sacred.Choose the log file you want to recover, use the notebookhelper/RL_results_repack.ipynbto convert the log file into.csvfile.Use the notebookRL Visualization Helper - Highway.ipynb(Heterogeneous Highway) orRL Visualization Helper - MPE.ipynb(Non-cooperative Navigation) to plot the reward curve
from the generated.csvfiles for each approaches and scenarios.Citation@article{wu2023iplan,
  title={iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning},
  author={Wu, Xiyang and Chandra, Rohan and Guan, Tianrui and Bedi, Amrit Singh and Manocha, Dinesh},
  journal={arXiv preprint arXiv:2306.06236},
  year={2023}
}