Package: gym-d2d
GymD2D: A Device-to-Device (D2D) Underlay Cellular Offload Evaluation PlatformGymD2D is a toolkit for building, evaluating and comparing D2D cellular offload resource allocation algorithms.It usesOpenAI Gymto make it easy to experiment with many popular reinforcement learning or AI algorithms.
It is highly configurable, allowing users to experiment with UE configuration, path loss and traffic models.GymD2D models a D2D cellular offload scenario containing a single macro base station surrounded with many cellular (CUE) and D2D (DUE) user equipment.This project is still under active development and we haven't finished the first stable release yet.Some functionality, such asenv.render()are not currently working.If you have found this project useful in your research, please consider citing ourwhite paper(preprint, forthcoming atIEEE WCNC 2021).@article{cotton2021gymd2d,title={GymD2D: A Device-to-Device Underlay Cellular Offload Evaluation Platform},author={Cotton, David and Chaczko, Zenon},journal={arXiv preprint arXiv:2101.11188},year={2021}}ContentsRequirementsInstallationDev InstallationUsageConfigurationEnvironment configurationDevice ConfigurationObservations and Rewards ConfigurationExamplesRequirementsPython 3.7 or greater (dataclasses&__future__.annotations)OpenAI Gym 0.9.6 or greater (env_config)NumPyInstallationUse pip to installpip install gym-d2dDev InstallationOr, if you need to edit the source code, clone and install dev dependencies:git clone git@github.com:davidcotton/gym-d2d.git
cd gym-d2d
pip install -e ".[dev]"UsageImport OpenAI Gym and GymD2Dimport gym
import gym_d2dBuild a new D2D environment via the usual Gym factory methodenv = gym.make('D2DEnv-v0')Then run in the standard Gym observation, action, reward loop.obses = env.reset()
game_over = False
while not game_over:
    actions = {}
    for agent_id, obs in obses.items():
        action = env.action_space['due'].sample()  # or: action = agent.act(obs)
        actions[agent_id] = action

    obses, rewards, game_over, infos = env.step(actions)
    env.render()The main difference between this environment and the usual classic control or ALE environments is that it is designed for multiple agents.
The environment's observation and action spaces usegym.spaces.DictSpace, with 3 keys:due,cue&mbs.
Observations, actions, rewards and info are passed via Python dicts like:obs_dict = {
    'cue00': np.ndarray(...),
    'cue01': np.ndarray(...),
    'due00': np.ndarray(...),
    'due01': np.ndarray(...),
    ...
}
actions = {
    'cue00': 23,
    'cue01': 317,
    'due00': 13,
    'due01': 95,
    ...
}We have some common usage examples in theexamples directory.ConfigurationOne of the design principles of this project is that environments should be easily configurable and customisable to meet the variety of research needs present in D2D cellular offload research.Environment configurationFollowing the Gym API (gym>=0.9.6), you can configure the environment via an env_config dictionary.env = gym.make('D2DEnv-v0', env_config={'param': value})The environment has the following configuration options:ParameterDescriptionDatatypeDefault Valuenum_rbsThe number of available resource blocks.int25num_cuesThe number of cellular users.int25num_due_pairsThe number of D2D pairsint25cell_radius_mThe macro base station's cell radius in metres. This parameter controls the radius in which all other devices are contained.float500.0d2d_radius_mThe maximum distance between D2D pairs in metres.float20.0due_min_tx_power_dBmThe minimum DUE transmission power in dBm.int0due_max_tx_power_dBmThe maximum DUE transmission power in dBm.int20cue_max_tx_power_dBmThe maximum CUE transmission power in dBm.int23mbs_max_tx_power_dBmThe maximum MBS transmission power in dBm.int46path_loss_modelThe type of path loss model to use.gym_d2d.PathLossgym_d2d.LogDistancePathLosstraffic_modelThe model to generate automated traffic.gym_d2d.TrafficModelgym_d2d.UplinkTrafficModelobs_fnThe function to calculate agent observations.gym_d2d.envs.ObsFunctiongym_d2d.envs.LinearObsFunctionreward_fnThe function to calculate agent rewards.gym_d2d.envs.RewardFunctiongym_d2d.envs.SystemCapacityRewardFunctioncarrier_freq_GHzThe carrier frequency used, in GHz.float2.1num_subcarriersThe number of subcarriers.int12subcarrier_spacing_kHzThe spacing between subcarriers.int15channel_bandwidth_MHzThe channel bandwidth in MHz.float20.0device_config_fileA path to a device configuration JSON file.pathlib.PathNone (random device positions)Device ConfigurationBy default, each time the environment isreset(), each UE is randomly assigned a new position.
To make experiments repeatable and compare algorithms, you may wish to fix UE positions.
This can be achieved by saving and loading device configurations.
Gym D2D uses Python'sPathlibto make file handling easier.from pathlib import PathTo save an environment's device configuration to file:env = gym.make('D2DEnv-v0')
env.reset()  # generate random device positions (if not supplied)
env.save_device_config(Path.cwd() / 'device_config.json')To load from an existing configuration:env_config = {'device_config_file': Path.cwd() / 'device_config.json'}
env = gym.make('D2DEnv-v0', env_config=env_config)Observations and Rewards ConfigurationMore info coming soon on how to customise observations and rewards...