Package: gymize
Gymize - Gym Reinforcement Learning with Unity 3DUnity and Python Reinforcement and Imitation Learning with Gymnasium and PettingZoo API.InstallationsInstallation for Python 3pip install gymizeIf you want to render videos, please installffmpegadditionally.You can also install ffmpeg with Anaconda:conda install -c conda-forge ffmpegInstallation for UnityGo to Window -> Package Manager -> Add package from git URL...Then paste the following git URL:https://github.com/timcsy/gymize.git?path=/unityIf git hasn't been installed, then you can download gymize first and install it from disk, specify the path to/unity/package.json, or you can install from tarball, downloaded from Releases.UsagesUsage for Python 3Assign theenv_name, which should be same as in the Unity. For example,kart.file_nameis the path to the built Unity game, or leaveNoneif using Unity Editor.Define the observation space in the format ofGym Spaces.Define the action space in the format ofGym Spaces.Assignrender_mode='video'if you want to record video from Unity, otherwise omitrender_mode.Assignviews=['', ...]if you want to record video from Unity, which given a list of view names, the empty string will be the default view, otherwise omitviews.Make the Gymnasium or PettingZoo environment by the following commands:Single-Agent with Gymnasium API:import gymnasium as gym
import gymize

env = gym.make(
    'gymize/Unity-v0',
    env_name='<your env name>',
    file_name=file_name,
    observation_space=observation_space,
    action_space=action_space,
    render_mode='<render_mode>',
    views=['', ...]
)Multi-Agents with PettingZoo AEC API:from gymize.envs import UnityAECEnv

env = UnityAECEnv(
    env_name='<your env name>',
    file_name=file_name,
    observation_spaces=observation_spaces,
    action_spaces=action_spaces,
    render_mode='<render_mode>',
    views=['', ...]
)Multi-Agents with PettingZoo Parallel API:from gymize.envs import UnityParallelEnv

env = UnityParallelEnv(
    env_name='<your env name>',
    file_name=file_name,
    observation_spaces=observation_spaces,
    action_spaces=action_spaces,
    render_mode='<render_mode>',
    views=['', ...]
)Well done! Now you can use the environment as the gym environment!The environmentenvwill have some additional methods other than Gymnasium or PettingZoo:env.unwrapped.send_info(info, agent=None)At anytime, you can send information throughinfoparameter in the form of Gymize Instance (see below) to Unity side.Theagentparameter is the agent name that will receive info (byGymize.Agent.OnInfo()method in the Unity side), orNonefor the environment to receive info (byGymize.GymEnv.OnInfo += (info) => {}listener in the Unity side).env.unwrapped.begin_render(screen_width=-1, screen_height=-1, fullscreen=False)Begin to record video in the Unity.screen_widthandscreen_heightis to set the width and height of the window, leave-1if you want the default window size.fullscreenis to set whether Unity is fullscreen.env.unwrapped.end_render()Stop to record video in the Unity.env.unwrapped.render_all(video_paths={})Render the videos which name and path are given by key value pairs invideo_path.UsingNoneas path for binary video object.env.unwrapped.render()orenv.render()Only render the default video, and return a binary video object.If you want to open the signaling service only, you can rungymize-signalingat the command line.Usage for UnityCreate Gym Manager ComponentAdd a game object in the scene.Add aGym ManagerComponent to the game object.Fill in theEnv Nameproperty with the name of the environment, which should be same as in the Python. For example,kart.Implement a class that inherit theAgentclass:class MyAgent : Agent {}Note that theAgentclass is inheritted fromMonoBehaviourclass.Override these listenerspublic override void OnReset() {}public override void OnAction(object obj) {}public override void OnInfo(object obj) {}AddTerminate(),Truncate()at the proper place.For example:public class MyAgent : Agent
{
    [Obs]
    private float pi = 3.14159f;

    private int m_Count;
    private float m_Speed;
    private string m_NickName;

    public override void OnReset()
    {
        // Reload the Unity scene when getting the reset signal from Python
        SceneManager.LoadScene(SceneManager.GetActiveScene().name, LoadSceneMode.Single);
    }

    public override void OnAction(object action)
    {
        Dictionary<string, object> actions = action as Dictionary<string, object>;
        List<object> arr = actions["arr"] as List<object>;

        Debug.Log((long)arr[0]); // arr[0] is Discrete value
        m_Count = Convert.ToInt32(arr[0]); // arr[0] is Discrete value
        m_Speed = Convert.ToSingle(actions["speed"]); // actions["speed"] is float value
    }

    public override void OnInfo(object info)
    {
        m_NickName = (string)info;
    }

    void Update()
    {
        // Terminate the game if collision occurred
        if (m_Collision)
        {
            // This method will tell the Python side to terminate the env
            Terminate();
        }
    }
}Add this Agent Component to the game object, and fill in theNameproperty with the name of the agent, usingagentfor single agent.Add ObserversReferbelowto learn more about the concept of "Locator".Using Attributes in the Agent class, assign the Locator or use the default Locator, for example:[Obs] // Using the default Locator ".UsedTime", which is same as the field name.
private float UsedTime;

[Obs(".Progress")]
private float m_Progress;

[Box]
private float Distance;

[Box(".Height=$")]
private float m_Height;You can useObsto use the default Gym space type depending on the source variable, or you can useBox(for Tensor, including MultiBinrary and MultiDiscrete),Discrete,Text,Dict,List,Graphattributes, see more details in theReflectionfolder, check outTestAgentInstance.csfor more examples.Add Sensor Component in the scene.Inheritted fromSensorComponentclass.CameraSensoris an example of a sensor component.Make sure that thepublic override IInstance GetObservation() {}method is implemented in the Sensor Component.Assign proper Locator in the Unity Editor.Add Gym Render Recorder Component to the scene if neededTheNameproperty can be empty or the name of the view.At the Python side, setrender_mode='video'if you want to render videos.You can disable theGym Managercomponent in the Unity Editor to develop the game without Python connection and play the game manually, it is useful for debugging.!!! Remember to close the channel in MonoBehaviour.OnApplicationQuit !!!Gymize InstanceThe instance generated from the action, observation space or info is called "Gymize Instance".Gymize Instance is defined in thespace.proto, which describes how the Gymize exchange data between Unity and Python, usingProtocol Buffers 3. Most of which originates from theGym Spaces.In Unity, check outGymInstance.csfor more information about how to convert the object into a meaningful type. In Python, you can treat the instance as usual object.Fundamental InstanceTensor: numpy array, with dtype and shapeCorresponding to the Gym Spaces asBox,MultiBinrary,MultiDiscreteDiscrete: int64Corresponding to the Gym Spaces asDiscreteText: stringCorresponding to the Gym Spaces asTextComposite InstanceDict:key,valuepairs mapping. Type ofkeyis string,valueis Gymize InstanceCorresponding to the Gym Spaces asDictList: array of Gymize InstanceCorresponding to the Gym Spaces asTuple,SequenceGraph: including three tensor objects, which arenodes,edgesandedge_linksnodes: the numeric information of nodesedges: the numeric information of edgesedge_links: the list of edges represent by node pairs (the node index begins with 0)Corresponding to the Gym Spaces asGraphAdditional Instance (Not defined in Gym Spaces)Raw: binary dataImage: image data, include format (PNG, JPG, ...), binary data, dtype, shape, axis permutationFloat: double precision floating numberBoolean: boolean value (true/false)JSON: JSON after stringifyingLocator"Locator" maps the observations collected from the Unity side to the specified location of Python side observation data, which is transferred by Gymize Instance.The followings are valid examples:example 1:.UsedTime = $example 2:.Progressexample 3:@.Raysexample 4:agent1@agent2@.key.0[12]["camera"]['front'][right][87](2)example 5:@@agent3@agent4@["camera"](1:10:2) = $(24:29) & @[11]=$[0] & @.key = $(3:8)For more examples, check outTestLocator.csandTestAgentInstance.cs.Structure of LocatorALocatoris a sequence ofMappings.AMappingconsists ofAgent,Destination, andSource.Agenthas four kinds: "all agents", "list agents", "root agent", and "omitted".DestinationandSourceare in the form ofSelector.ASelectorcan act on the following types:Dict,Tuple,Sequence,Tensor."key" selector is forDict."index" or "slices" selector is forTuple,Sequence,Tensor.ASliceis Python-like or Numpy-like, which has "start", "stop", and "step".Usually we writestart:stop:step, e.g.1:10:2.There are some special cases: "index", "ellipsis", and "new axis".Syntax for LocatorSyntax forLocatorUsing&to connect differentMappings.e.g..field1 & .field2=$ & @.field3=$["key"]Syntax ofMapping{Agent}Selectors(Destination){=$Selectors(Source)}{ } means can be omitted.If you omitAgent, then the Locator will become relative w.r.t Agent.If you omit=$Selectors(Source), it is same as=$.=means mapping or assignment. It will map the source (right hand side) to the destination (left hand side).$means the Unity side observation (variable or sensor data).Syntax forAgentUsingagent@to assign the agent name.Usingagent1@agent2@to assign the agent names.Using@@means for all agents.Using@@agent3@agent4@means for all agents, exceptagent3andagent4.Using@to represent the root agent itself.You can omit theAgentto use relative location.Syntax forSelectorForDict:.key,['key'],["key"], or[key]ForTuple:[index]or[slice]ForSequence:[], means appendForTensor:(Slice)or(Slice1, Slice2, ..., SliceN)The severalTensorselectors have to put at the end of the selector sequence.Syntax forSliceSame syntax as Python or numpy.start:stop:step, e.g.1:10:2.omitted "step", it becomes1:10.omitted "stop", it becomes1:.omitted "start", it becomes:.You can just use an integer to represent the index (negative integer means counting from the end).You can use...to represent ellipsis.You can usenewaxisornp.newaxisto represent new axis.Seelocator.bnffor more information about the syntax.Known issuesGymize get the observation space type information by generating a sample instance, so it may not get the type information ofSequenceif it samples a empty array.On the Python side, it may cause some problems if you edit the observation data directly, because it is not a copy, it is a reference object.Dict source with different keys but with same destination Locator may not merged correctly, check outspace.pyfor more details.If you run the built Unity Application and it fails after first OnReset, try:It may be this issue:ArgumentNullException: Value cannot be null. Parameter name: shader.Go to: Edit -> ProjectSettings -> GraphicsChange the size of Always Included Shaders, and add theRuntime/Space/Grayscale.shaderinto it.