Repository: sdpkjc/abcdrl (language: python)

Issue:
build(deps): bump autorom from 0.4.2 to 0.5.0
Bumps [autorom](https://github.com/Farama-Foundation/AutoROM) from 0.4.2 to 0.5.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/Farama-Foundation/AutoROM/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=autorom&package-manager=pip&previous-version=0.4.2&new-version=0.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>


Discussion / hints:

Fix patch:
diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml
index 8effe9e..058c23d 100644
--- a/.github/workflows/pre-commit.yml
+++ b/.github/workflows/pre-commit.yml
@@ -2,7 +2,7 @@ name: pre-commit
 on: [push]
 jobs:
   build:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
     steps:
       - uses: actions/checkout@v2
         with:
@@ -14,4 +14,4 @@ jobs:
           python-version: 3.9
       - uses: pre-commit/action@v2.0.3
         with:
-          extra_args: --hook-stage manual --all-files
+          extra_args: --all-files
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 64cdd38..24ce83e 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -53,6 +53,7 @@ repos:
     - id: poetry-export
       name: poetry-export requirements-dev.txt
       args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-dev.txt", "-E", "all", "--with", "dev"]
+    # torch
     - id: poetry-export
       name: poetry-export requirements-torch.txt
       args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-torch.txt", "-E", "torch", "-E", "atari", "-E", "mujoco"]
@@ -60,14 +61,45 @@ repos:
       name: poetry-export requirements-torch-dev.txt
       args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-torch-dev.txt", "-E", "torch", "-E", "atari", "-E", "mujoco", "--with", "dev"]
     - id: poetry-export
-      name: poetry-export requirements-torch-atari.txt
-      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-torch-atari.txt", "-E", "torch", "-E", "atari"]
+      name: poetry-export requirements-basic-torch.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-basic-torch.txt", "-E", "torch"]
     - id: poetry-export
-      name: poetry-export requirements-torch-atari-dev.txt
-      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-torch-atari-dev.txt", "-E", "torch", "-E", "atari", "--with", "dev"]
+      name: poetry-export requirements-basic-torch-dev.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-basic-torch-dev.txt", "-E", "torch", "--with", "dev"]
     - id: poetry-export
-      name: poetry-export requirements-torch-mujoco.txt
-      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-torch-mujoco.txt", "-E", "torch", "-E", "mujoco"]
+      name: poetry-export requirements-atari-torch.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-atari-torch.txt", "-E", "torch", "-E", "atari"]
     - id: poetry-export
-      name: poetry-export requirements-torch-mujoco-dev.txt
-      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-torch-mujoco-dev.txt", "-E", "torch", "-E", "mujoco", "--with", "dev"]
+      name: poetry-export requirements-atari-torch-dev.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-atari-torch-dev.txt", "-E", "torch", "-E", "atari", "--with", "dev"]
+    - id: poetry-export
+      name: poetry-export requirements-mujoco-torch.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-mujoco-torch.txt", "-E", "torch", "-E", "mujoco"]
+    - id: poetry-export
+      name: poetry-export requirements-mujoco-torch-dev.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-mujoco-torch-dev.txt", "-E", "torch", "-E", "mujoco", "--with", "dev"]
+    # tensorflow2
+    - id: poetry-export
+      name: poetry-export requirements-tf.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-tf.txt", "-E", "tf", "-E", "atari", "-E", "mujoco"]
+    - id: poetry-export
+      name: poetry-export requirements-tf-dev.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-tf-dev.txt", "-E", "tf", "-E", "atari", "-E", "mujoco", "--with", "dev"]
+    - id: poetry-export
+      name: poetry-export requirements-basic-tf.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-basic-tf.txt", "-E", "tf"]
+    - id: poetry-export
+      name: poetry-export requirements-basic-tf-dev.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-basic-tf-dev.txt", "-E", "tf", "--with", "dev"]
+    - id: poetry-export
+      name: poetry-export requirements-atari-tf.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-atari-tf.txt", "-E", "tf", "-E", "atari"]
+    - id: poetry-export
+      name: poetry-export requirements-atari-tf-dev.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-atari-tf-dev.txt", "-E", "tf", "-E", "atari", "--with", "dev"]
+    - id: poetry-export
+      name: poetry-export requirements-mujoco-tf.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-mujoco-tf.txt", "-E", "tf", "-E", "mujoco"]
+    - id: poetry-export
+      name: poetry-export requirements-mujoco-tf-dev.txt
+      args: ["--without-hashes", "--without-urls", "-o", "requirements/requirements-mujoco-tf-dev.txt", "-E", "tf", "-E", "mujoco", "--with", "dev"]
diff --git a/README.cn.md b/README.cn.md
index 5f40f92..618a968 100644
--- a/README.cn.md
+++ b/README.cn.md
@@ -22,7 +22,7 @@ abcdRL 是一个**模块化单文件强化学习代码库**，提供“有但不
 
 *阅读代码时，在单文件代码中，快速了解算法的完整实现细节；改进算法时，得益于轻量的模块化设计，只需专注于少量的模块。*
 
-> abcdRL 主要参考了 [vwxyzjn/cleanrL](https://github.com/vwxyzjn/cleanrl/) 的单文件设计哲学和 [PaddlePaddle/PARL](https://github.com/PaddlePaddle/PARL/) 的模块设计。
+> abcdRL 主要参考了 [vwxyzjn/cleanrl](https://github.com/vwxyzjn/cleanrl/) 的单文件设计哲学和 [PaddlePaddle/PARL](https://github.com/PaddlePaddle/PARL/) 的模块设计。
 
 ***使用文档 ➡️ [docs.abcdrl.xyz](https://docs.abcdrl.xyz/zh/)***
 
@@ -39,7 +39,7 @@ abcdRL 是一个**模块化单文件强化学习代码库**，提供“有但不
 ```shell
 # 0. 安装 Docker & Nvidia Drive & NVIDIA Container Toolkit
 # 1. 运行 DQN 算法
-docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn.py
+docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn_torch.py
 ```
 
 ***[详细安装说明 👀](https://docs.abcdrl.xyz/zh/install/)***
@@ -64,16 +64,16 @@ docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn.py
 
 ***Weights & Biases 性能报告 ➡️ [report.abcdrl.xyz](https://report.abcdrl.xyz)***
 
-- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236)
-- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971)
-- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477)
-- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290)
-- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477)
+- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236) <sub>`dqn_torch.py`, `dqn_tf.py`, `dqn_atari_torch.py`, `dqn_atari_tf.py`</sub>
+- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971) <sub>`ddpg_torch.py`</sub>
+- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477) <sub>`td3_torch.py`</sub>
+- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290) <sub>`sac_torch.py`</sub>
+- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477) <sub>`ppo_torch.py`</sub>
 
 ---
 
-- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461)
-- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952)
+- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461) <sub>`ddqn_torch.py`, `ddqn_tf.py`</sub>
+- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952) <sub>`pdqn_torch.py`, `pdqn_tf.py`</sub>
 
 ## 引用 abcdRL
 
diff --git a/README.md b/README.md
index 4f3d7c9..6514e08 100644
--- a/README.md
+++ b/README.md
@@ -24,7 +24,7 @@ abcdRL is a **Modular Single-file Reinforcement Learning Algorithms Library** th
 
 > abcdRL mainly references the single-file design philosophy of [vwxyzjn/cleanrl](https://github.com/vwxyzjn/cleanrl/) and the module design of [PaddlePaddle/PARL](https://github.com/PaddlePaddle/PARL/).
 
-***Documentation ➡️ [docs.abcdrl.xyz](https://abcdrl.xyz)***
+***Documentation ➡️ [docs.abcdrl.xyz](https://docs.abcdrl.xyz/)***
 
 ***Roadmap🗺️ [#57](https://github.com/sdpkjc/abcdrl/issues/57)***
 
@@ -39,7 +39,7 @@ Using Docker📦:
 ```shell
 # 0. Prerequisites: Docker & Nvidia Drive & NVIDIA Container Toolkit
 # 1. Run DQN algorithm
-docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn.py
+docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn_torch.py
 ```
 
 ***[For detailed installation instructions 👀](https://docs.abcdrl.xyz/install/)***
@@ -64,16 +64,16 @@ docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn.py
 
 ***Weights & Biases Benchmark Report ➡️ [report.abcdrl.xyz](https://report.abcdrl.xyz)***
 
-- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236)
-- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971)
-- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477)
-- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290)
-- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477)
+- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236) <sub>`dqn_torch.py`, `dqn_tf.py`, `dqn_atari_torch.py`, `dqn_atari_tf.py`</sub>
+- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971) <sub>`ddpg_torch.py`</sub>
+- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477) <sub>`td3_torch.py`</sub>
+- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290) <sub>`sac_torch.py`</sub>
+- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477) <sub>`ppo_torch.py`</sub>
 
 ---
 
-- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461)
-- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952)
+- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461) <sub>`ddqn_torch.py`, `ddqn_tf.py`</sub>
+- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952) <sub>`pdqn_torch.py`, `pdqn_tf.py`</sub>
 
 ## Citing abcdRL
 
diff --git a/abcdrl/__init__.py b/abcdrl/__init__.py
index 2036afe..e69de29 100644
--- a/abcdrl/__init__.py
+++ b/abcdrl/__init__.py
@@ -1,8 +0,0 @@
-import abcdrl.ddpg  # noqa
-import abcdrl.ddqn  # noqa
-import abcdrl.dqn  # noqa
-import abcdrl.dqn_atari  # noqa
-import abcdrl.pdqn  # noqa
-import abcdrl.ppo  # noqa
-import abcdrl.sac  # noqa
-import abcdrl.td3  # noqa
diff --git a/abcdrl/ddpg.py b/abcdrl/ddpg_torch.py
similarity index 95%
rename from abcdrl/ddpg.py
rename to abcdrl/ddpg_torch.py
index 9b99f91..3a90dc0 100644
--- a/abcdrl/ddpg.py
+++ b/abcdrl/ddpg_torch.py
@@ -15,7 +15,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -65,8 +64,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -252,7 +251,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "Hopper-v4",
         num_envs: int = 1,
@@ -275,11 +274,11 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["policy_frequency"] = max(
             self.kwargs["policy_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Box)
@@ -345,7 +344,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env.action_space.seed(self.kwargs["seed"] + idx)
             env.observation_space.seed(self.kwargs["seed"] + idx)
             return env
@@ -353,10 +352,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -380,7 +380,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -388,12 +387,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -410,15 +409,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/ddqn_tf.py b/abcdrl/ddqn_tf.py
new file mode 100644
index 0000000..53e19a1
--- /dev/null
+++ b/abcdrl/ddqn_tf.py
@@ -0,0 +1,394 @@
+from __future__ import annotations
+
+import copy
+import dataclasses
+import os
+import random
+import time
+from typing import Any, Callable, Generator, Generic, TypeVar
+
+import fire
+import gymnasium as gym
+import numpy as np
+import tensorflow as tf
+from combine_signatures.combine_signatures import combine_signatures
+from tensorflow.keras import layers, losses, models, optimizers
+
+SamplesItemType = TypeVar("SamplesItemType", tf.Tensor, np.ndarray)
+
+
+def get_space_shape(env_space: gym.Space) -> tuple[int, ...]:
+    if isinstance(env_space, gym.spaces.Box):
+        return env_space.shape
+    elif isinstance(env_space, gym.spaces.Discrete):
+        return (1,)
+    elif isinstance(env_space, gym.spaces.MultiDiscrete):
+        return (int(len(env_space.nvec)),)
+    elif isinstance(env_space, gym.spaces.MultiBinary):
+        if type(env_space.n) in [tuple, list, np.ndarray]:
+            return tuple(env_space.n)
+        else:
+            return (int(env_space.n),)
+    raise NotImplementedError(f"{env_space} observation space is not supported")
+
+
+class ReplayBuffer:
+    @dataclasses.dataclass(frozen=True)
+    class Samples(Generic[SamplesItemType]):
+        observations: SamplesItemType
+        actions: SamplesItemType
+        next_observations: SamplesItemType
+        dones: SamplesItemType
+        rewards: SamplesItemType
+
+    def __init__(
+        self,
+        obs_space: gym.Space,
+        act_space: gym.Space,
+        buffer_size: int = 1_000_0,
+    ) -> None:
+        self.obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        self.next_obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        self.acts_buf = np.zeros((buffer_size,) + get_space_shape(act_space), dtype=act_space.dtype)
+        self.rews_buf = np.zeros((buffer_size,), dtype=np.float32)
+        self.dones_buf = np.zeros((buffer_size,), dtype=np.float32)
+
+        self.buffer_size = buffer_size
+        self.ptr = 0
+        self.size = 0
+
+    def add(
+        self,
+        obs: np.ndarray,
+        next_obs: np.ndarray,
+        act: np.ndarray,
+        rew: np.ndarray,
+        done: np.ndarray,
+        infos: dict[str, Any],
+    ) -> None:
+        for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
+            self.obs_buf[self.ptr] = np.array(obs_i).copy()
+            self.next_obs_buf[self.ptr] = np.array(next_obs_i).copy()
+            self.acts_buf[self.ptr] = np.array(act_i).copy()
+            self.rews_buf[self.ptr] = np.array(rew_i).copy()
+            self.dones_buf[self.ptr] = np.array(done_i).copy()
+            self.ptr = (self.ptr + 1) % self.buffer_size
+            self.size = min(self.size + 1, self.buffer_size)
+
+    def sample(self, batch_size: int = 1) -> Samples[np.ndarray]:
+        idxs = np.random.choice(self.size, size=batch_size, replace=True)
+        return ReplayBuffer.Samples[np.ndarray](
+            observations=self.obs_buf[idxs],
+            next_observations=self.next_obs_buf[idxs],
+            actions=self.acts_buf[idxs],
+            rewards=self.rews_buf[idxs],
+            dones=self.dones_buf[idxs],
+        )
+
+    def __len__(self) -> int:
+        return self.size
+
+
+class Network(models.Model):
+    def __init__(self, out_n: int, name: str = "q_network", **kwargs):
+        super().__init__(name=name, **kwargs)
+        self.layer_dense_0 = layers.Dense(120, activation="relu")
+        self.layer_dense_1 = layers.Dense(84, activation="relu")
+        self.layer_output = layers.Dense(out_n)
+
+    def call(self, x: tf.Tensor) -> tf.Tensor:
+        x = self.layer_dense_0(x)
+        x = self.layer_dense_1(x)
+        x = self.layer_output(x)
+        return x
+
+
+class Model:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        # input shape: int(np.prod(get_space_shape(self.kwargs["obs_space"])))
+        self.network = Network(
+            self.kwargs["act_space"].n,
+        )
+
+    def value(self, obs: tf.Tensor) -> tf.Tensor:
+        return self.network(obs)
+
+
+class Algorithm:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.model = Model(**self.kwargs)
+        self.model_t = copy.deepcopy(self.model)
+        self.optimizer = optimizers.Adam(self.kwargs["learning_rate"])
+        self.loss_func = losses.MeanSquaredError()
+
+        model_init_obs = tf.convert_to_tensor(np.array([self.kwargs["obs_space"].sample()]))
+        self.model.value(model_init_obs)
+        self.model_t.value(model_init_obs)
+
+    def predict(self, obs: tf.Tensor) -> tf.Tensor:
+        val = self.model.value(obs)
+        return val
+
+    def learn(self, data: ReplayBuffer.Samples[tf.Tensor]) -> dict[str, Any]:
+        # double dqn
+        target_next_q_val = self.model_t.value(data.next_observations)
+        target_argmax = tf.math.argmax(target_next_q_val, axis=1)
+        target_max = tf.squeeze(
+            tf.gather(
+                target_next_q_val,
+                indices=tf.squeeze(target_argmax),
+                batch_dims=tf.rank(tf.squeeze(target_argmax)),
+                axis=1,
+            )
+        )
+        td_target = data.rewards + self.kwargs["gamma"] * target_max * (1 - data.dones)
+
+        with tf.GradientTape() as tape:
+            old_val = tf.squeeze(
+                tf.gather(
+                    self.model.value(data.observations),
+                    indices=tf.squeeze(data.actions),
+                    batch_dims=tf.rank(tf.squeeze(data.actions)),
+                    axis=1,
+                )
+            )
+            td_loss = self.loss_func(td_target, old_val)
+        grads = tape.gradient(td_loss, self.model.network.trainable_variables)
+        self.optimizer.apply_gradients(zip(grads, self.model.network.trainable_variables))
+
+        log_data = {"td_loss": td_loss, "q_value": tf.reduce_mean(old_val)}
+        return log_data
+
+    def sync_target(self) -> None:
+        self.model_t.network.set_weights(self.model.network.get_weights())
+
+
+class Agent:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.alg = Algorithm(**self.kwargs)
+        self.sample_step = 0
+        self.learn_step = 0
+
+    def predict(self, obs: np.ndarray) -> np.ndarray:
+        obs_ts = tf.convert_to_tensor(obs)
+        act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+        act_np = act_ts.numpy()
+        return act_np
+
+    def sample(self, obs: np.ndarray) -> np.ndarray:
+        if random.random() < self._get_epsilon():
+            act_np = np.array([self.kwargs["act_space"].sample() for _ in range(self.kwargs["num_envs"])])
+        else:
+            obs_ts = tf.convert_to_tensor(obs)
+            act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+            act_np = act_ts.numpy()
+
+        self.sample_step += self.kwargs["num_envs"]
+        if self.sample_step % self.kwargs["target_network_frequency"] == 0:
+            self.alg.sync_target()
+        return act_np
+
+    def learn(self, data: ReplayBuffer.Samples[np.ndarray]) -> dict[str, Any]:
+        data_ts = ReplayBuffer.Samples[tf.Tensor](
+            **{
+                item[0]: tf.convert_to_tensor(item[1]) if isinstance(item[1], np.ndarray) else item[1]
+                for item in dataclasses.asdict(data).items()
+            }
+        )
+
+        log_data = self.alg.learn(data_ts)
+        self.learn_step += 1
+        log_data["epsilon"] = self._get_epsilon()
+        return log_data
+
+    def _get_epsilon(self) -> float:
+        slope = (self.kwargs["end_epsilon"] - self.kwargs["start_epsilon"]) * (
+            self.sample_step / (self.kwargs["exploration_fraction"] * self.kwargs["total_timesteps"])
+        ) + self.kwargs["start_epsilon"]
+        return max(slope, self.kwargs["end_epsilon"])
+
+
+class Trainer:
+    def __init__(
+        self,
+        exp_name: str | None = None,
+        seed: int = 1,
+        cuda: bool = True,
+        capture_video: bool = False,
+        env_id: str = "CartPole-v1",
+        num_envs: int = 1,
+        total_timesteps: int = 500_000,
+        gamma: float = 0.99,
+        # Collect
+        buffer_size: int = 10_000,
+        start_epsilon: float = 1.0,
+        end_epsilon: float = 0.05,
+        exploration_fraction: float = 0.5,
+        # Learn
+        batch_size: int = 128,
+        learning_rate: float = 2.5e-4,
+        # Train
+        learning_starts: int = 10_000,
+        target_network_frequency: int = 500,
+        train_frequency: int = 10,
+    ) -> None:
+        self.kwargs = locals()
+        self.kwargs.pop("self")
+
+        if self.kwargs["exp_name"] is None:
+            self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
+        self.kwargs["target_network_frequency"] = max(
+            self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
+        )
+        if not self.kwargs["cuda"]:
+            tf.config.experimental.set_visible_devices([], "GPU")
+
+        self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
+        assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
+
+        self.kwargs["obs_space"] = self.envs.single_observation_space
+        self.kwargs["act_space"] = self.envs.single_action_space
+
+        self.buffer = ReplayBuffer(
+            self.kwargs["obs_space"],
+            self.kwargs["act_space"],
+            buffer_size=self.kwargs["buffer_size"],
+        )
+
+        self.obs, _ = self.envs.reset(seed=self.kwargs["seed"])
+        self.agent = Agent(**self.kwargs)
+
+    def __call__(self) -> Generator[dict[str, Any], None, None]:
+        for _ in range(self.kwargs["learning_starts"]):
+            yield self._run_collect()
+        while self.agent.sample_step < self.kwargs["total_timesteps"]:
+            for _ in range(self.kwargs["train_frequency"]):
+                if not self.agent.sample_step < self.kwargs["total_timesteps"]:
+                    break
+                yield self._run_collect()
+            yield self._run_train()
+
+        self.envs.close_extras()
+
+    def _run_collect(self) -> dict[str, Any]:
+        act = self.agent.sample(self.obs)
+        next_obs, reward, terminated, truncated, infos = self.envs.step(act)
+
+        real_next_obs = next_obs.copy()
+        if "final_observation" in infos.keys():
+            for idx, final_obs in enumerate(infos["final_observation"]):
+                real_next_obs[idx] = real_next_obs[idx] if final_obs is None else final_obs
+
+        self.buffer.add(self.obs, real_next_obs, act, reward, terminated, infos)
+        self.obs = next_obs
+
+        if "final_info" in infos.keys():
+            final_info = next(item for item in infos["final_info"] if item is not None)
+            return {
+                "log_type": "collect",
+                "sample_step": self.agent.sample_step,
+                "logs": {
+                    "episodic_length": final_info["episode"]["l"][0],
+                    "episodic_return": final_info["episode"]["r"][0],
+                },
+            }
+        return {"log_type": "collect", "sample_step": self.agent.sample_step}
+
+    def _run_train(self) -> dict[str, Any]:
+        data = self.buffer.sample(batch_size=self.kwargs["batch_size"])
+        log_data = self.agent.learn(data)
+
+        return {"log_type": "train", "sample_step": self.agent.sample_step, "logs": log_data}
+
+    def _make_env(self, idx: int) -> Callable[[], gym.Env]:
+        def thunk() -> gym.Env:
+            env = gym.make(self.kwargs["env_id"], render_mode="rgb_array")
+            env = gym.wrappers.RecordEpisodeStatistics(env)
+            if self.kwargs["capture_video"]:
+                if idx == 0:
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
+            env.action_space.seed(self.kwargs["seed"] + idx)
+            env.observation_space.seed(self.kwargs["seed"] + idx)
+            return env
+
+        return thunk
+
+
+def wrapper_logger_tf(
+    wrapped: Callable[..., Generator[dict[str, Any], None, None]]
+) -> Callable[..., Generator[dict[str, Any], None, None]]:
+    import wandb
+
+    def setup_video_monitor() -> None:
+        vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
+        vcr.close_ = vcr.close  # type: ignore[attr-defined]
+
+        def close(self):
+            vcr.close_(self)
+            if self.path:
+                wandb.log({"videos": wandb.Video(self.path)})
+                self.path = None
+
+        vcr.close = close  # type: ignore[assignment]
+
+    @combine_signatures(wrapped)
+    def _wrapper(
+        *args,
+        track: bool = False,
+        wandb_project_name: str = "abcdrl",
+        wandb_tags: list[str] = [],
+        wandb_entity: str | None = None,
+        **kwargs,
+    ) -> Generator[dict[str, Any], None, None]:
+        instance = args[0]
+        if track:
+            wandb.init(
+                project=wandb_project_name,
+                tags=wandb_tags,
+                entity=wandb_entity,
+                sync_tensorboard=True,
+                config=instance.kwargs,
+                name=instance.kwargs["run_name"],
+                save_code=True,
+            )
+            setup_video_monitor()
+
+        writer = tf.summary.create_file_writer(f"runs/{instance.kwargs['run_name']}")
+        with writer.as_default():
+            tf.summary.text(
+                "hyperparameters",
+                "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
+                0,
+            )
+
+            gen = wrapped(*args, **kwargs)
+            for log_data in gen:
+                if "logs" in log_data:
+                    for log_item in log_data["logs"].items():
+                        tf.summary.scalar(f"{log_data['log_type']}/{log_item[0]}", log_item[1], log_data["sample_step"])
+                yield log_data
+
+    return _wrapper
+
+
+if __name__ == "__main__":
+    SEED = 1234
+    os.environ["PYTHONHASHSEED"] = str(SEED)
+    os.environ["TF_DETERMINISTIC_OPS"] = "1"
+    os.environ["TF_CUDNN_DETERMINISTIC"] = "1"
+    random.seed(SEED)
+    np.random.seed(SEED)
+    tf.random.set_seed(SEED)
+
+    Trainer.__call__ = wrapper_logger_tf(Trainer.__call__)  # type: ignore[assignment]
+    fire.Fire(
+        Trainer,
+        serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
+    )
diff --git a/abcdrl/ddqn.py b/abcdrl/ddqn_torch.py
similarity index 94%
rename from abcdrl/ddqn.py
rename to abcdrl/ddqn_torch.py
index b7d4b47..b1ef610 100644
--- a/abcdrl/ddqn.py
+++ b/abcdrl/ddqn_torch.py
@@ -15,7 +15,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -65,8 +64,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -210,7 +209,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "CartPole-v1",
         num_envs: int = 1,
@@ -234,11 +233,11 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["target_network_frequency"] = max(
             self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
@@ -303,7 +302,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env.action_space.seed(self.kwargs["seed"] + idx)
             env.observation_space.seed(self.kwargs["seed"] + idx)
             return env
@@ -311,10 +310,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -338,7 +338,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -346,12 +345,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -368,15 +367,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/dqn_atari_tf.py b/abcdrl/dqn_atari_tf.py
new file mode 100644
index 0000000..a32458d
--- /dev/null
+++ b/abcdrl/dqn_atari_tf.py
@@ -0,0 +1,513 @@
+from __future__ import annotations
+
+import copy
+import dataclasses
+import os
+import random
+import time
+from typing import Any, Callable, Generator, Generic, TypeVar
+
+import fire
+import gymnasium as gym
+import numpy as np
+import tensorflow as tf
+from combine_signatures.combine_signatures import combine_signatures
+from tensorflow.keras import layers, losses, models, optimizers
+
+SamplesItemType = TypeVar("SamplesItemType", tf.Tensor, np.ndarray)
+
+
+def get_space_shape(env_space: gym.Space) -> tuple[int, ...]:
+    if isinstance(env_space, gym.spaces.Box):
+        return env_space.shape
+    elif isinstance(env_space, gym.spaces.Discrete):
+        return (1,)
+    elif isinstance(env_space, gym.spaces.MultiDiscrete):
+        return (int(len(env_space.nvec)),)
+    elif isinstance(env_space, gym.spaces.MultiBinary):
+        if type(env_space.n) in [tuple, list, np.ndarray]:
+            return tuple(env_space.n)
+        else:
+            return (int(env_space.n),)
+    raise NotImplementedError(f"{env_space} observation space is not supported")
+
+
+class NoopResetEnv(gym.Wrapper):
+    def __init__(self, env: gym.Env, noop_max: int = 30) -> None:
+        gym.Wrapper.__init__(self, env)
+        self.noop_max = noop_max
+        self.override_num_noops = None
+        self.noop_action = 0
+        assert env.unwrapped.get_action_meanings()[0] == "NOOP"  # type: ignore[attr-defined]
+
+    def reset(self, **kwargs) -> tuple[np.ndarray, dict[str, Any]]:
+        self.env.reset(**kwargs)
+        if self.override_num_noops is not None:
+            noops = self.override_num_noops
+        else:
+            noops = self.unwrapped.np_random.integers(1, self.noop_max + 1)
+        assert noops > 0
+
+        for _ in range(noops):
+            obs, _, terminated, truncated, info = self.env.step(self.noop_action)
+            if terminated or truncated:
+                obs, info = self.env.reset(**kwargs)
+        return obs, info
+
+
+class FireResetEnv(gym.Wrapper):
+    def __init__(self, env: gym.Env) -> None:
+        gym.Wrapper.__init__(self, env)
+        assert env.unwrapped.get_action_meanings()[1] == "FIRE"  # type: ignore[attr-defined]
+        assert len(env.unwrapped.get_action_meanings()) >= 3  # type: ignore[attr-defined]
+
+    def reset(self, **kwargs) -> tuple[np.ndarray, dict[str, Any]]:
+        self.env.reset(**kwargs)
+        obs, _, terminated, truncated, info = self.env.step(1)
+        if terminated or truncated:
+            self.env.reset(**kwargs)
+        obs, _, terminated, truncated, info = self.env.step(2)
+        if terminated or truncated:
+            self.env.reset(**kwargs)
+        return obs, info
+
+
+class EpisodicLifeEnv(gym.Wrapper):
+    def __init__(self, env: gym.Env) -> None:
+        gym.Wrapper.__init__(self, env)
+        self.lives = 0
+        self.was_real_done = True
+
+    def step(self, action: int) -> tuple[np.ndarray, float, bool, bool, dict[str, Any]]:
+        obs, reward, terminated, truncated, info = self.env.step(action)
+        self.was_real_done = terminated or truncated
+        lives = self.env.unwrapped.ale.lives()  # type: ignore[attr-defined]
+        if 0 < lives < self.lives:
+            terminated = True
+        self.lives = lives
+        return obs, reward, terminated, truncated, info
+
+    def reset(self, **kwargs) -> tuple[np.ndarray, dict[str, Any]]:
+        if self.was_real_done:
+            obs, info = self.env.reset(**kwargs)
+        else:
+            obs, _, _, _, info = self.env.step(0)
+        self.lives = self.env.unwrapped.ale.lives()  # type: ignore[attr-defined]
+        return obs, info
+
+
+class MaxAndSkipEnv(gym.Wrapper):
+    def __init__(self, env: gym.Env, skip: int = 4) -> None:
+        gym.Wrapper.__init__(self, env)
+        self._obs_buffer = np.zeros((2,) + env.observation_space.shape, dtype=env.observation_space.dtype)
+        self._skip = skip
+
+    def step(self, action: int) -> tuple[np.ndarray, float, bool, bool, dict[str, Any]]:
+        total_reward = 0.0
+        terminated, truncated = None, None
+        for i in range(self._skip):
+            obs, reward, terminated, truncated, info = self.env.step(action)
+            if i == self._skip - 2:
+                self._obs_buffer[0] = obs
+            if i == self._skip - 1:
+                self._obs_buffer[1] = obs
+            total_reward += reward
+            if terminated or truncated:
+                break
+        max_frame = self._obs_buffer.max(axis=0)
+
+        return max_frame, total_reward, terminated, truncated, info
+
+
+class ReplayBuffer:
+    @dataclasses.dataclass(frozen=True)
+    class Samples(Generic[SamplesItemType]):
+        observations: SamplesItemType
+        actions: SamplesItemType
+        next_observations: SamplesItemType
+        dones: SamplesItemType
+        rewards: SamplesItemType
+
+    def __init__(
+        self,
+        obs_space: gym.Space,
+        act_space: gym.Space,
+        buffer_size: int = 1_000_0,
+        optimize_memory_usage: bool = False,
+    ) -> None:
+        self.obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        if not optimize_memory_usage:
+            self.next_obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        self.acts_buf = np.zeros((buffer_size,) + get_space_shape(act_space), dtype=act_space.dtype)
+        self.rews_buf = np.zeros((buffer_size,), dtype=np.float32)
+        self.dones_buf = np.zeros((buffer_size,), dtype=np.float32)
+
+        self.buffer_size = buffer_size
+        self.ptr = 0
+        self.size = 0
+        self.optimize_memory_usage = optimize_memory_usage
+
+    def add(
+        self,
+        obs: np.ndarray,
+        next_obs: np.ndarray,
+        act: np.ndarray,
+        rew: np.ndarray,
+        done: np.ndarray,
+        infos: dict[str, Any],
+    ) -> None:
+        for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
+            self.obs_buf[self.ptr] = np.array(obs_i).copy()
+            if not self.optimize_memory_usage:
+                self.next_obs_buf[self.ptr] = np.array(next_obs_i).copy()
+            else:
+                self.obs_buf[(self.ptr + 1) % self.buffer_size] = np.array(next_obs_i).copy()
+            self.acts_buf[self.ptr] = np.array(act_i).copy()
+            self.rews_buf[self.ptr] = np.array(rew_i).copy()
+            self.dones_buf[self.ptr] = np.array(done_i).copy()
+            self.ptr = (self.ptr + 1) % self.buffer_size
+            self.size = min(self.size + 1, self.buffer_size)
+
+    def sample(self, batch_size: int = 1) -> Samples[np.ndarray]:
+        if not self.optimize_memory_usage:
+            idxs = np.random.choice(self.size, size=batch_size, replace=True)
+            next_observations = self.next_obs_buf[idxs]
+        else:
+            if self.size != self.buffer_size:
+                idxs = np.random.choice(self.size, size=batch_size, replace=True)
+            else:
+                idxs = (
+                    (np.random.choice(self.size - 1, size=batch_size, replace=True) + 1) + self.ptr
+                ) % self.buffer_size
+            next_observations = self.obs_buf[(idxs + 1) % self.buffer_size]
+
+        return ReplayBuffer.Samples[np.ndarray](
+            observations=self.obs_buf[idxs],
+            next_observations=next_observations,
+            actions=self.acts_buf[idxs],
+            rewards=self.rews_buf[idxs],
+            dones=self.dones_buf[idxs],
+        )
+
+    def __len__(self) -> int:
+        return self.size
+
+
+class Network(models.Model):
+    def __init__(self, out_n: int, name: str = "q_network", **kwargs):
+        super().__init__(name=name, **kwargs)
+        self.layer_permute = layers.Permute((2, 3, 1))
+        self.layer_cov_0 = layers.Conv2D(32, 8, strides=4, activation="relu", input_shape=(4, 84, 84))
+        self.layer_cov_1 = layers.Conv2D(64, 4, strides=2, activation="relu")
+        self.layer_cov_2 = layers.Conv2D(64, 4, strides=1, activation="relu")
+        self.layer_flatten_3 = layers.Flatten()
+        self.layer_dense_4 = layers.Dense(512, activation="relu")
+        self.layer_output = layers.Dense(out_n)
+
+    def call(self, x: tf.Tensor) -> tf.Tensor:
+        x = self.layer_permute(x)
+        x = self.layer_cov_0(x)
+        x = self.layer_cov_1(x)
+        x = self.layer_cov_2(x)
+        x = self.layer_flatten_3(x)
+        x = self.layer_dense_4(x)
+        x = self.layer_output(x)
+        return x
+
+
+class Model:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.network = Network(
+            self.kwargs["act_space"].n,
+        )
+
+    def value(self, obs: tf.Tensor) -> tf.Tensor:
+        return self.network(obs / 255.0)
+
+
+class Algorithm:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.model = Model(**self.kwargs)
+        self.model_t = copy.deepcopy(self.model)
+        self.optimizer = optimizers.Adam(self.kwargs["learning_rate"])
+        self.loss_func = losses.MeanSquaredError()
+
+        model_init_obs = tf.convert_to_tensor(np.array([self.kwargs["obs_space"].sample()]), dtype=tf.float32)
+        self.model.value(model_init_obs)
+        self.model_t.value(model_init_obs)
+
+    def predict(self, obs: tf.Tensor) -> tf.Tensor:
+        val = self.model.value(obs)
+        return val
+
+    def learn(self, data: ReplayBuffer.Samples[tf.Tensor]) -> dict[str, Any]:
+        target_max = tf.math.reduce_max(self.model_t.value(data.next_observations), axis=1)
+        td_target = data.rewards + self.kwargs["gamma"] * target_max * (1 - data.dones)
+
+        with tf.GradientTape() as tape:
+            old_val = tf.squeeze(
+                tf.gather(
+                    self.model.value(data.observations),
+                    indices=tf.squeeze(data.actions),
+                    batch_dims=tf.rank(tf.squeeze(data.actions)),
+                    axis=1,
+                )
+            )
+            td_loss = self.loss_func(td_target, old_val)
+        grads = tape.gradient(td_loss, self.model.network.trainable_variables)
+        self.optimizer.apply_gradients(zip(grads, self.model.network.trainable_variables))
+
+        log_data = {"td_loss": td_loss, "q_value": tf.reduce_mean(old_val)}
+        return log_data
+
+    def sync_target(self) -> None:
+        self.model_t.network.set_weights(self.model.network.get_weights())
+
+
+class Agent:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.alg = Algorithm(**self.kwargs)
+        self.sample_step = 0
+        self.learn_step = 0
+
+    def predict(self, obs: np.ndarray) -> np.ndarray:
+        obs_ts = tf.convert_to_tensor(obs, dtype=tf.float32)
+        act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+        act_np = act_ts.numpy()
+        return act_np
+
+    def sample(self, obs: np.ndarray) -> np.ndarray:
+        if random.random() < self._get_epsilon():
+            act_np = np.array([self.kwargs["act_space"].sample() for _ in range(self.kwargs["num_envs"])])
+        else:
+            obs_ts = tf.convert_to_tensor(obs, dtype=tf.float32)
+            act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+            act_np = act_ts.numpy()
+
+        self.sample_step += self.kwargs["num_envs"]
+        if self.sample_step % self.kwargs["target_network_frequency"] == 0:
+            self.alg.sync_target()
+        return act_np
+
+    def learn(self, data: ReplayBuffer.Samples[np.ndarray]) -> dict[str, Any]:
+        data_ts = ReplayBuffer.Samples[tf.Tensor](
+            **{
+                item[0]: tf.convert_to_tensor(item[1]) if isinstance(item[1], np.ndarray) else item[1]
+                for item in dataclasses.asdict(data).items()
+            }
+        )
+        data_ts = dataclasses.replace(
+            data_ts,
+            observations=tf.cast(data_ts.observations, dtype=tf.float32),
+            next_observations=tf.cast(data_ts.next_observations, dtype=tf.float32),
+        )
+
+        log_data = self.alg.learn(data_ts)
+        self.learn_step += 1
+        log_data["epsilon"] = self._get_epsilon()
+        return log_data
+
+    def _get_epsilon(self) -> float:
+        slope = (self.kwargs["end_epsilon"] - self.kwargs["start_epsilon"]) * (
+            self.sample_step / (self.kwargs["exploration_fraction"] * self.kwargs["total_timesteps"])
+        ) + self.kwargs["start_epsilon"]
+        return max(slope, self.kwargs["end_epsilon"])
+
+
+class Trainer:
+    def __init__(
+        self,
+        exp_name: str | None = None,
+        seed: int = 1,
+        cuda: bool = True,
+        capture_video: bool = False,
+        env_id: str = "BreakoutNoFrameskip-v4",
+        num_envs: int = 1,
+        total_timesteps: int = 10_000_000,
+        gamma: float = 0.99,
+        # Collect
+        buffer_size: int = 1_000_000,
+        start_epsilon: float = 1.0,
+        end_epsilon: float = 0.01,
+        exploration_fraction: float = 0.1,
+        # Learn
+        batch_size: int = 32,
+        learning_rate: float = 1e-4,
+        # Train
+        learning_starts: int = 50_000,
+        target_network_frequency: int = 1000,
+        train_frequency: int = 4,
+    ) -> None:
+        self.kwargs = locals()
+        self.kwargs.pop("self")
+
+        if self.kwargs["exp_name"] is None:
+            self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
+        self.kwargs["target_network_frequency"] = max(
+            self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
+        )
+        if not self.kwargs["cuda"]:
+            tf.config.experimental.set_visible_devices([], "GPU")
+
+        self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
+        assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
+
+        self.kwargs["obs_space"] = self.envs.single_observation_space
+        self.kwargs["act_space"] = self.envs.single_action_space
+
+        self.buffer = ReplayBuffer(
+            self.kwargs["obs_space"],
+            self.kwargs["act_space"],
+            buffer_size=self.kwargs["buffer_size"],
+            optimize_memory_usage=True,
+        )
+
+        self.obs, _ = self.envs.reset(seed=self.kwargs["seed"])
+        self.agent = Agent(**self.kwargs)
+
+    def __call__(self) -> Generator[dict[str, Any], None, None]:
+        for _ in range(self.kwargs["learning_starts"]):
+            yield self._run_collect()
+        while self.agent.sample_step < self.kwargs["total_timesteps"]:
+            for _ in range(self.kwargs["train_frequency"]):
+                if not self.agent.sample_step < self.kwargs["total_timesteps"]:
+                    break
+                yield self._run_collect()
+            yield self._run_train()
+
+        self.envs.close_extras()
+
+    def _run_collect(self) -> dict[str, Any]:
+        act = self.agent.sample(self.obs)
+        next_obs, reward, terminated, truncated, infos = self.envs.step(act)
+
+        real_next_obs = next_obs.copy()
+        if "final_observation" in infos.keys():
+            for idx, final_obs in enumerate(infos["final_observation"]):
+                real_next_obs[idx] = real_next_obs[idx] if final_obs is None else final_obs
+
+        self.buffer.add(self.obs, real_next_obs, act, reward, terminated, infos)
+        self.obs = next_obs
+
+        if "final_info" in infos.keys():
+            final_info = next(item for item in infos["final_info"] if item is not None)
+            # EpisodicLifeEnv
+            if "episode" in final_info:
+                return {
+                    "log_type": "collect",
+                    "sample_step": self.agent.sample_step,
+                    "logs": {
+                        "episodic_length": final_info["episode"]["l"][0],
+                        "episodic_return": final_info["episode"]["r"][0],
+                    },
+                }
+        return {"log_type": "collect", "sample_step": self.agent.sample_step}
+
+    def _run_train(self) -> dict[str, Any]:
+        data = self.buffer.sample(batch_size=self.kwargs["batch_size"])
+        log_data = self.agent.learn(data)
+
+        return {"log_type": "train", "sample_step": self.agent.sample_step, "logs": log_data}
+
+    def _make_env(self, idx: int) -> Callable[[], gym.Env]:
+        def thunk() -> gym.Env:
+            env = gym.make(self.kwargs["env_id"], render_mode="rgb_array")
+            env = gym.wrappers.RecordEpisodeStatistics(env)
+            if self.kwargs["capture_video"]:
+                if idx == 0:
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
+            if "NOOP" in env.unwrapped.get_action_meanings():  # type: ignore[attr-defined]
+                env = NoopResetEnv(env, noop_max=30)
+            env = MaxAndSkipEnv(env, skip=4)
+            env = EpisodicLifeEnv(env)
+            if "FIRE" in env.unwrapped.get_action_meanings():  # type: ignore[attr-defined]
+                env = FireResetEnv(env)
+            env = gym.wrappers.TransformReward(env, np.sign)
+            env = gym.wrappers.ResizeObservation(env, (84, 84))
+            env = gym.wrappers.GrayScaleObservation(env)
+            env = gym.wrappers.FrameStack(env, 4)
+            env.action_space.seed(self.kwargs["seed"] + idx)
+            env.observation_space.seed(self.kwargs["seed"] + idx)
+            return env
+
+        return thunk
+
+
+def wrapper_logger_tf(
+    wrapped: Callable[..., Generator[dict[str, Any], None, None]]
+) -> Callable[..., Generator[dict[str, Any], None, None]]:
+    import wandb
+
+    def setup_video_monitor() -> None:
+        vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
+        vcr.close_ = vcr.close  # type: ignore[attr-defined]
+
+        def close(self):
+            vcr.close_(self)
+            if self.path:
+                wandb.log({"videos": wandb.Video(self.path)})
+                self.path = None
+
+        vcr.close = close  # type: ignore[assignment]
+
+    @combine_signatures(wrapped)
+    def _wrapper(
+        *args,
+        track: bool = False,
+        wandb_project_name: str = "abcdrl",
+        wandb_tags: list[str] = [],
+        wandb_entity: str | None = None,
+        **kwargs,
+    ) -> Generator[dict[str, Any], None, None]:
+        instance = args[0]
+        if track:
+            wandb.init(
+                project=wandb_project_name,
+                tags=wandb_tags,
+                entity=wandb_entity,
+                sync_tensorboard=True,
+                config=instance.kwargs,
+                name=instance.kwargs["run_name"],
+                save_code=True,
+            )
+            setup_video_monitor()
+
+        writer = tf.summary.create_file_writer(f"runs/{instance.kwargs['run_name']}")
+        with writer.as_default():
+            tf.summary.text(
+                "hyperparameters",
+                "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
+                0,
+            )
+
+            gen = wrapped(*args, **kwargs)
+            for log_data in gen:
+                if "logs" in log_data:
+                    for log_item in log_data["logs"].items():
+                        tf.summary.scalar(f"{log_data['log_type']}/{log_item[0]}", log_item[1], log_data["sample_step"])
+                yield log_data
+
+    return _wrapper
+
+
+if __name__ == "__main__":
+    SEED = 1234
+    os.environ["PYTHONHASHSEED"] = str(SEED)
+    os.environ["TF_DETERMINISTIC_OPS"] = "1"
+    os.environ["TF_CUDNN_DETERMINISTIC"] = "1"
+    random.seed(SEED)
+    np.random.seed(SEED)
+    tf.random.set_seed(SEED)
+
+    Trainer.__call__ = wrapper_logger_tf(Trainer.__call__)  # type: ignore[assignment]
+    fire.Fire(
+        Trainer,
+        serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
+    )
diff --git a/abcdrl/dqn_atari.py b/abcdrl/dqn_atari_torch.py
similarity index 95%
rename from abcdrl/dqn_atari.py
rename to abcdrl/dqn_atari_torch.py
index 6fb172b..2a8f0dd 100644
--- a/abcdrl/dqn_atari.py
+++ b/abcdrl/dqn_atari_torch.py
@@ -15,7 +15,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -155,8 +154,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -213,7 +212,7 @@ def __init__(self, out_n: int) -> None:
         )
 
     def forward(self, x: torch.Tensor) -> torch.Tensor:
-        return self.network(x / 255.0)
+        return self.network(x)
 
 
 class Model(nn.Module):
@@ -226,7 +225,7 @@ def __init__(self, **kwargs) -> None:
         )
 
     def value(self, obs: torch.Tensor) -> torch.Tensor:
-        return self.network(obs)
+        return self.network(obs / 255.0)
 
 
 class Algorithm:
@@ -316,7 +315,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "BreakoutNoFrameskip-v4",
         num_envs: int = 1,
@@ -340,11 +339,11 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["target_network_frequency"] = max(
             self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
@@ -412,7 +411,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             if "NOOP" in env.unwrapped.get_action_meanings():  # type: ignore[attr-defined]
                 env = NoopResetEnv(env, noop_max=30)
             env = MaxAndSkipEnv(env, skip=4)
@@ -430,10 +429,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -457,7 +457,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -465,12 +464,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -487,15 +486,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/dqn_tf.py b/abcdrl/dqn_tf.py
new file mode 100644
index 0000000..b2c0e39
--- /dev/null
+++ b/abcdrl/dqn_tf.py
@@ -0,0 +1,384 @@
+from __future__ import annotations
+
+import copy
+import dataclasses
+import os
+import random
+import time
+from typing import Any, Callable, Generator, Generic, TypeVar
+
+import fire
+import gymnasium as gym
+import numpy as np
+import tensorflow as tf
+from combine_signatures.combine_signatures import combine_signatures
+from tensorflow.keras import layers, losses, models, optimizers
+
+SamplesItemType = TypeVar("SamplesItemType", tf.Tensor, np.ndarray)
+
+
+def get_space_shape(env_space: gym.Space) -> tuple[int, ...]:
+    if isinstance(env_space, gym.spaces.Box):
+        return env_space.shape
+    elif isinstance(env_space, gym.spaces.Discrete):
+        return (1,)
+    elif isinstance(env_space, gym.spaces.MultiDiscrete):
+        return (int(len(env_space.nvec)),)
+    elif isinstance(env_space, gym.spaces.MultiBinary):
+        if type(env_space.n) in [tuple, list, np.ndarray]:
+            return tuple(env_space.n)
+        else:
+            return (int(env_space.n),)
+    raise NotImplementedError(f"{env_space} observation space is not supported")
+
+
+class ReplayBuffer:
+    @dataclasses.dataclass(frozen=True)
+    class Samples(Generic[SamplesItemType]):
+        observations: SamplesItemType
+        actions: SamplesItemType
+        next_observations: SamplesItemType
+        dones: SamplesItemType
+        rewards: SamplesItemType
+
+    def __init__(
+        self,
+        obs_space: gym.Space,
+        act_space: gym.Space,
+        buffer_size: int = 1_000_0,
+    ) -> None:
+        self.obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        self.next_obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        self.acts_buf = np.zeros((buffer_size,) + get_space_shape(act_space), dtype=act_space.dtype)
+        self.rews_buf = np.zeros((buffer_size,), dtype=np.float32)
+        self.dones_buf = np.zeros((buffer_size,), dtype=np.float32)
+
+        self.buffer_size = buffer_size
+        self.ptr = 0
+        self.size = 0
+
+    def add(
+        self,
+        obs: np.ndarray,
+        next_obs: np.ndarray,
+        act: np.ndarray,
+        rew: np.ndarray,
+        done: np.ndarray,
+        infos: dict[str, Any],
+    ) -> None:
+        for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
+            self.obs_buf[self.ptr] = np.array(obs_i).copy()
+            self.next_obs_buf[self.ptr] = np.array(next_obs_i).copy()
+            self.acts_buf[self.ptr] = np.array(act_i).copy()
+            self.rews_buf[self.ptr] = np.array(rew_i).copy()
+            self.dones_buf[self.ptr] = np.array(done_i).copy()
+            self.ptr = (self.ptr + 1) % self.buffer_size
+            self.size = min(self.size + 1, self.buffer_size)
+
+    def sample(self, batch_size: int = 1) -> Samples[np.ndarray]:
+        idxs = np.random.choice(self.size, size=batch_size, replace=True)
+        return ReplayBuffer.Samples[np.ndarray](
+            observations=self.obs_buf[idxs],
+            next_observations=self.next_obs_buf[idxs],
+            actions=self.acts_buf[idxs],
+            rewards=self.rews_buf[idxs],
+            dones=self.dones_buf[idxs],
+        )
+
+    def __len__(self) -> int:
+        return self.size
+
+
+class Network(models.Model):
+    def __init__(self, out_n: int, name: str = "q_network", **kwargs):
+        super().__init__(name=name, **kwargs)
+        self.layer_dense_0 = layers.Dense(120, activation="relu")
+        self.layer_dense_1 = layers.Dense(84, activation="relu")
+        self.layer_output = layers.Dense(out_n)
+
+    def call(self, x: tf.Tensor) -> tf.Tensor:
+        x = self.layer_dense_0(x)
+        x = self.layer_dense_1(x)
+        x = self.layer_output(x)
+        return x
+
+
+class Model:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        # input shape: int(np.prod(get_space_shape(self.kwargs["obs_space"])))
+        self.network = Network(
+            self.kwargs["act_space"].n,
+        )
+
+    def value(self, obs: tf.Tensor) -> tf.Tensor:
+        return self.network(obs)
+
+
+class Algorithm:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.model = Model(**self.kwargs)
+        self.model_t = copy.deepcopy(self.model)
+        self.optimizer = optimizers.Adam(self.kwargs["learning_rate"])
+        self.loss_func = losses.MeanSquaredError()
+
+        model_init_obs = tf.convert_to_tensor(np.array([self.kwargs["obs_space"].sample()]))
+        self.model.value(model_init_obs)
+        self.model_t.value(model_init_obs)
+
+    def predict(self, obs: tf.Tensor) -> tf.Tensor:
+        val = self.model.value(obs)
+        return val
+
+    def learn(self, data: ReplayBuffer.Samples[tf.Tensor]) -> dict[str, Any]:
+        target_max = tf.math.reduce_max(self.model_t.value(data.next_observations), axis=1)
+        td_target = data.rewards + self.kwargs["gamma"] * target_max * (1 - data.dones)
+
+        with tf.GradientTape() as tape:
+            old_val = tf.squeeze(
+                tf.gather(
+                    self.model.value(data.observations),
+                    indices=tf.squeeze(data.actions),
+                    batch_dims=tf.rank(tf.squeeze(data.actions)),
+                    axis=1,
+                )
+            )
+            td_loss = self.loss_func(td_target, old_val)
+        grads = tape.gradient(td_loss, self.model.network.trainable_variables)
+        self.optimizer.apply_gradients(zip(grads, self.model.network.trainable_variables))
+
+        log_data = {"td_loss": td_loss, "q_value": tf.reduce_mean(old_val)}
+        return log_data
+
+    def sync_target(self) -> None:
+        self.model_t.network.set_weights(self.model.network.get_weights())
+
+
+class Agent:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.alg = Algorithm(**self.kwargs)
+        self.sample_step = 0
+        self.learn_step = 0
+
+    def predict(self, obs: np.ndarray) -> np.ndarray:
+        obs_ts = tf.convert_to_tensor(obs)
+        act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+        act_np = act_ts.numpy()
+        return act_np
+
+    def sample(self, obs: np.ndarray) -> np.ndarray:
+        if random.random() < self._get_epsilon():
+            act_np = np.array([self.kwargs["act_space"].sample() for _ in range(self.kwargs["num_envs"])])
+        else:
+            obs_ts = tf.convert_to_tensor(obs)
+            act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+            act_np = act_ts.numpy()
+
+        self.sample_step += self.kwargs["num_envs"]
+        if self.sample_step % self.kwargs["target_network_frequency"] == 0:
+            self.alg.sync_target()
+        return act_np
+
+    def learn(self, data: ReplayBuffer.Samples[np.ndarray]) -> dict[str, Any]:
+        data_ts = ReplayBuffer.Samples[tf.Tensor](
+            **{
+                item[0]: tf.convert_to_tensor(item[1]) if isinstance(item[1], np.ndarray) else item[1]
+                for item in dataclasses.asdict(data).items()
+            }
+        )
+
+        log_data = self.alg.learn(data_ts)
+        self.learn_step += 1
+        log_data["epsilon"] = self._get_epsilon()
+        return log_data
+
+    def _get_epsilon(self) -> float:
+        slope = (self.kwargs["end_epsilon"] - self.kwargs["start_epsilon"]) * (
+            self.sample_step / (self.kwargs["exploration_fraction"] * self.kwargs["total_timesteps"])
+        ) + self.kwargs["start_epsilon"]
+        return max(slope, self.kwargs["end_epsilon"])
+
+
+class Trainer:
+    def __init__(
+        self,
+        exp_name: str | None = None,
+        seed: int = 1,
+        cuda: bool = True,
+        capture_video: bool = False,
+        env_id: str = "CartPole-v1",
+        num_envs: int = 1,
+        total_timesteps: int = 500_000,
+        gamma: float = 0.99,
+        # Collect
+        buffer_size: int = 10_000,
+        start_epsilon: float = 1.0,
+        end_epsilon: float = 0.05,
+        exploration_fraction: float = 0.5,
+        # Learn
+        batch_size: int = 128,
+        learning_rate: float = 2.5e-4,
+        # Train
+        learning_starts: int = 10_000,
+        target_network_frequency: int = 500,
+        train_frequency: int = 10,
+    ) -> None:
+        self.kwargs = locals()
+        self.kwargs.pop("self")
+
+        if self.kwargs["exp_name"] is None:
+            self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
+        self.kwargs["target_network_frequency"] = max(
+            self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
+        )
+        if not self.kwargs["cuda"]:
+            tf.config.experimental.set_visible_devices([], "GPU")
+
+        self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
+        assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
+
+        self.kwargs["obs_space"] = self.envs.single_observation_space
+        self.kwargs["act_space"] = self.envs.single_action_space
+
+        self.buffer = ReplayBuffer(
+            self.kwargs["obs_space"],
+            self.kwargs["act_space"],
+            buffer_size=self.kwargs["buffer_size"],
+        )
+
+        self.obs, _ = self.envs.reset(seed=self.kwargs["seed"])
+        self.agent = Agent(**self.kwargs)
+
+    def __call__(self) -> Generator[dict[str, Any], None, None]:
+        for _ in range(self.kwargs["learning_starts"]):
+            yield self._run_collect()
+        while self.agent.sample_step < self.kwargs["total_timesteps"]:
+            for _ in range(self.kwargs["train_frequency"]):
+                if not self.agent.sample_step < self.kwargs["total_timesteps"]:
+                    break
+                yield self._run_collect()
+            yield self._run_train()
+
+        self.envs.close_extras()
+
+    def _run_collect(self) -> dict[str, Any]:
+        act = self.agent.sample(self.obs)
+        next_obs, reward, terminated, truncated, infos = self.envs.step(act)
+
+        real_next_obs = next_obs.copy()
+        if "final_observation" in infos.keys():
+            for idx, final_obs in enumerate(infos["final_observation"]):
+                real_next_obs[idx] = real_next_obs[idx] if final_obs is None else final_obs
+
+        self.buffer.add(self.obs, real_next_obs, act, reward, terminated, infos)
+        self.obs = next_obs
+
+        if "final_info" in infos.keys():
+            final_info = next(item for item in infos["final_info"] if item is not None)
+            return {
+                "log_type": "collect",
+                "sample_step": self.agent.sample_step,
+                "logs": {
+                    "episodic_length": final_info["episode"]["l"][0],
+                    "episodic_return": final_info["episode"]["r"][0],
+                },
+            }
+        return {"log_type": "collect", "sample_step": self.agent.sample_step}
+
+    def _run_train(self) -> dict[str, Any]:
+        data = self.buffer.sample(batch_size=self.kwargs["batch_size"])
+        log_data = self.agent.learn(data)
+
+        return {"log_type": "train", "sample_step": self.agent.sample_step, "logs": log_data}
+
+    def _make_env(self, idx: int) -> Callable[[], gym.Env]:
+        def thunk() -> gym.Env:
+            env = gym.make(self.kwargs["env_id"], render_mode="rgb_array")
+            env = gym.wrappers.RecordEpisodeStatistics(env)
+            if self.kwargs["capture_video"]:
+                if idx == 0:
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
+            env.action_space.seed(self.kwargs["seed"] + idx)
+            env.observation_space.seed(self.kwargs["seed"] + idx)
+            return env
+
+        return thunk
+
+
+def wrapper_logger_tf(
+    wrapped: Callable[..., Generator[dict[str, Any], None, None]]
+) -> Callable[..., Generator[dict[str, Any], None, None]]:
+    import wandb
+
+    def setup_video_monitor() -> None:
+        vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
+        vcr.close_ = vcr.close  # type: ignore[attr-defined]
+
+        def close(self):
+            vcr.close_(self)
+            if self.path:
+                wandb.log({"videos": wandb.Video(self.path)})
+                self.path = None
+
+        vcr.close = close  # type: ignore[assignment]
+
+    @combine_signatures(wrapped)
+    def _wrapper(
+        *args,
+        track: bool = False,
+        wandb_project_name: str = "abcdrl",
+        wandb_tags: list[str] = [],
+        wandb_entity: str | None = None,
+        **kwargs,
+    ) -> Generator[dict[str, Any], None, None]:
+        instance = args[0]
+        if track:
+            wandb.init(
+                project=wandb_project_name,
+                tags=wandb_tags,
+                entity=wandb_entity,
+                sync_tensorboard=True,
+                config=instance.kwargs,
+                name=instance.kwargs["run_name"],
+                save_code=True,
+            )
+            setup_video_monitor()
+
+        writer = tf.summary.create_file_writer(f"runs/{instance.kwargs['run_name']}")
+        with writer.as_default():
+            tf.summary.text(
+                "hyperparameters",
+                "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
+                0,
+            )
+
+            gen = wrapped(*args, **kwargs)
+            for log_data in gen:
+                if "logs" in log_data:
+                    for log_item in log_data["logs"].items():
+                        tf.summary.scalar(f"{log_data['log_type']}/{log_item[0]}", log_item[1], log_data["sample_step"])
+                yield log_data
+
+    return _wrapper
+
+
+if __name__ == "__main__":
+    SEED = 1234
+    os.environ["PYTHONHASHSEED"] = str(SEED)
+    os.environ["TF_DETERMINISTIC_OPS"] = "1"
+    os.environ["TF_CUDNN_DETERMINISTIC"] = "1"
+    random.seed(SEED)
+    np.random.seed(SEED)
+    tf.random.set_seed(SEED)
+
+    Trainer.__call__ = wrapper_logger_tf(Trainer.__call__)  # type: ignore[assignment]
+    fire.Fire(
+        Trainer,
+        serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
+    )
diff --git a/abcdrl/dqn.py b/abcdrl/dqn_torch.py
similarity index 94%
rename from abcdrl/dqn.py
rename to abcdrl/dqn_torch.py
index e1b6cf3..61ba974 100644
--- a/abcdrl/dqn.py
+++ b/abcdrl/dqn_torch.py
@@ -15,7 +15,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -65,8 +64,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -208,7 +207,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "CartPole-v1",
         num_envs: int = 1,
@@ -232,11 +231,11 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["target_network_frequency"] = max(
             self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
@@ -301,7 +300,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env.action_space.seed(self.kwargs["seed"] + idx)
             env.observation_space.seed(self.kwargs["seed"] + idx)
             return env
@@ -309,10 +308,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -336,7 +336,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -344,12 +343,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -366,15 +365,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/pdqn_tf.py b/abcdrl/pdqn_tf.py
new file mode 100644
index 0000000..c664509
--- /dev/null
+++ b/abcdrl/pdqn_tf.py
@@ -0,0 +1,503 @@
+from __future__ import annotations
+
+import copy
+import dataclasses
+import operator
+import os
+import random
+import time
+from typing import Any, Callable, Generator, Generic, TypeVar
+
+import fire
+import gymnasium as gym
+import numpy as np
+import tensorflow as tf
+from combine_signatures.combine_signatures import combine_signatures
+from tensorflow.keras import layers, losses, models, optimizers
+
+SamplesItemType = TypeVar("SamplesItemType", tf.Tensor, np.ndarray)
+
+
+def get_space_shape(env_space: gym.Space) -> tuple[int, ...]:
+    if isinstance(env_space, gym.spaces.Box):
+        return env_space.shape
+    elif isinstance(env_space, gym.spaces.Discrete):
+        return (1,)
+    elif isinstance(env_space, gym.spaces.MultiDiscrete):
+        return (int(len(env_space.nvec)),)
+    elif isinstance(env_space, gym.spaces.MultiBinary):
+        if type(env_space.n) in [tuple, list, np.ndarray]:
+            return tuple(env_space.n)
+        else:
+            return (int(env_space.n),)
+    raise NotImplementedError(f"{env_space} observation space is not supported")
+
+
+class PrioritizedReplayBuffer:
+    @dataclasses.dataclass(frozen=True)
+    class Samples(Generic[SamplesItemType]):
+        observations: SamplesItemType
+        actions: SamplesItemType
+        next_observations: SamplesItemType
+        dones: SamplesItemType
+        rewards: SamplesItemType
+        weights: SamplesItemType
+        indices: list[int]
+
+    class SegmentTree:
+        def __init__(self, capacity: int, operation: Callable[[float, float], float], init_value: float) -> None:
+            assert capacity > 0 and capacity % 2 == 0
+
+            self.capacity = capacity
+            self.operation = operation
+            self.init_value = init_value
+            self.tree = [init_value for _ in range(2 * capacity)]
+
+        def query(self, start: int, end: int) -> float:
+            return self._query(start, end, 1, 0, self.capacity)
+
+        def _query(self, start: int, end: int, node: int, node_start: int, node_end: int) -> float:
+            if start <= node_start and node_end <= end:
+                return self.tree[node]
+
+            mid = (node_start + node_end) // 2
+            return self.operation(
+                self._query(start, end, 2 * node, node_start, mid) if start <= mid else self.init_value,
+                self._query(start, end, 2 * node + 1, mid + 1, node_end) if end > mid else self.init_value,
+            )
+
+        def retrieve(self, upperbound: float) -> int:
+            assert self.operation == operator.add
+            assert 0 <= upperbound <= self.query(0, self.capacity - 1) + 1e-5
+
+            idx = 1
+            while idx < self.capacity:
+                idx *= 2
+                if self.tree[idx] <= upperbound:
+                    upperbound -= self.tree[idx]
+                    idx += 1
+
+            return idx - self.capacity
+
+        def __setitem__(self, idx: int, val: float) -> None:
+            idx += self.capacity
+            self.tree[idx] = val
+
+            while idx // 2:
+                idx //= 2
+                self.tree[idx] = self.operation(self.tree[2 * idx], self.tree[2 * idx + 1])
+
+        def __getitem__(self, idx: int) -> float:
+            return self.tree[self.capacity + idx]
+
+    def __init__(
+        self,
+        obs_space: gym.Space,
+        act_space: gym.Space,
+        buffer_size: int = 1_000_0,
+        alpha: float = 0.2,
+    ) -> None:
+        assert alpha >= 0
+
+        self.obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        self.next_obs_buf = np.zeros((buffer_size,) + get_space_shape(obs_space), dtype=obs_space.dtype)
+        self.acts_buf = np.zeros((buffer_size,) + get_space_shape(act_space), dtype=act_space.dtype)
+        self.rews_buf = np.zeros((buffer_size,), dtype=np.float32)
+        self.dones_buf = np.zeros((buffer_size,), dtype=np.float32)
+
+        self.buffer_size = buffer_size
+        self.ptr = 0
+        self.size = 0
+
+        self.max_priority, self.tree_ptr = 1.0, 0
+        self.alpha = alpha
+        tree_capacity = 1
+        while tree_capacity < self.buffer_size:
+            tree_capacity *= 2
+
+        self.sum_tree = self.SegmentTree(tree_capacity, operator.add, 0.0)
+        self.min_tree = self.SegmentTree(tree_capacity, min, float("inf"))
+
+    def add(
+        self,
+        obs: np.ndarray,
+        next_obs: np.ndarray,
+        act: np.ndarray,
+        rew: np.ndarray,
+        done: np.ndarray,
+        infos: dict[str, Any],
+    ) -> None:
+        for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
+            self.obs_buf[self.ptr] = np.array(obs_i).copy()
+            self.next_obs_buf[self.ptr] = np.array(next_obs_i).copy()
+            self.acts_buf[self.ptr] = np.array(act_i).copy()
+            self.rews_buf[self.ptr] = np.array(rew_i).copy()
+            self.dones_buf[self.ptr] = np.array(done_i).copy()
+            self.ptr = (self.ptr + 1) % self.buffer_size
+            self.size = min(self.size + 1, self.buffer_size)
+
+            self.sum_tree[self.tree_ptr] = self.max_priority**self.alpha
+            self.min_tree[self.tree_ptr] = self.max_priority**self.alpha
+            self.tree_ptr = (self.tree_ptr + 1) % self.buffer_size
+
+    def sample(self, batch_size: int = 1, beta: float = 0.6) -> Samples:
+        assert len(self) >= batch_size
+        assert beta > 0
+
+        indices = self._sample_proportional(batch_size)
+
+        obs = self.obs_buf[indices]
+        next_obs = self.next_obs_buf[indices]
+        acts = self.acts_buf[indices]
+        rews = self.rews_buf[indices]
+        done = self.dones_buf[indices]
+        weights = np.array([self._calculate_weight(i, beta) for i in indices], dtype=np.float32).reshape(-1, 1)
+
+        return PrioritizedReplayBuffer.Samples(
+            observations=obs,
+            next_observations=next_obs,
+            actions=acts,
+            rewards=rews,
+            dones=done,
+            weights=weights,
+            indices=indices,
+        )
+
+    def update_priorities(self, indices: list[int], priorities: np.ndarray) -> None:
+        assert len(indices) == len(priorities)
+
+        for idx, priority in zip(indices, priorities):
+            assert priority > 0
+            assert 0 <= idx < len(self)
+
+            self.sum_tree[idx] = priority**self.alpha
+            self.min_tree[idx] = priority**self.alpha
+            self.max_priority = max(self.max_priority, priority)
+
+    def __len__(self) -> int:
+        return self.size
+
+    def _sample_proportional(self, batch_size: int = 1) -> list[int]:
+        indices = []
+        p_total = self.sum_tree.query(0, len(self) - 1)
+        segment = p_total / batch_size
+
+        for i in range(batch_size):
+            upperbound = random.uniform(segment * i, segment * (i + 1))
+            idx = self.sum_tree.retrieve(upperbound)
+            indices.append(idx)
+
+        return indices
+
+    def _calculate_weight(self, idx: int, beta: float) -> float:
+        p_min = self.min_tree.query(0, len(self) - 1) / self.sum_tree.query(0, len(self) - 1)
+        max_weight = (p_min * len(self)) ** (-beta)
+
+        p_sample = self.sum_tree[idx] / self.sum_tree.query(0, len(self) - 1)
+        weight = (p_sample * len(self)) ** (-beta)
+        weight = weight / max_weight
+        return float(weight)
+
+
+class Network(models.Model):
+    def __init__(self, out_n: int, name: str = "q_network", **kwargs):
+        super().__init__(name=name, **kwargs)
+        self.layer_dense_0 = layers.Dense(120, activation="relu")
+        self.layer_dense_1 = layers.Dense(84, activation="relu")
+        self.layer_output = layers.Dense(out_n)
+
+    def call(self, x: tf.Tensor) -> tf.Tensor:
+        x = self.layer_dense_0(x)
+        x = self.layer_dense_1(x)
+        x = self.layer_output(x)
+        return x
+
+
+class Model:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        # input shape: int(np.prod(get_space_shape(self.kwargs["obs_space"])))
+        self.network = Network(
+            self.kwargs["act_space"].n,
+        )
+
+    def value(self, obs: tf.Tensor) -> tf.Tensor:
+        return self.network(obs)
+
+
+class Algorithm:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.model = Model(**self.kwargs)
+        self.model_t = copy.deepcopy(self.model)
+        self.optimizer = optimizers.Adam(self.kwargs["learning_rate"])
+        self.loss_func = losses.MeanSquaredError(reduction=tf.keras.losses.Reduction.NONE)
+
+        model_init_obs = tf.convert_to_tensor(np.array([self.kwargs["obs_space"].sample()]))
+        self.model.value(model_init_obs)
+        self.model_t.value(model_init_obs)
+
+    def predict(self, obs: tf.Tensor) -> tf.Tensor:
+        val = self.model.value(obs)
+        return val
+
+    def learn(self, data: PrioritizedReplayBuffer.Samples[tf.Tensor]) -> dict[str, Any]:
+        target_max = tf.math.reduce_max(self.model_t.value(data.next_observations), axis=1)
+        td_target = data.rewards + self.kwargs["gamma"] * target_max * (1 - data.dones)
+
+        with tf.GradientTape() as tape:
+            old_val = tf.squeeze(
+                tf.gather(
+                    self.model.value(data.observations),
+                    indices=tf.squeeze(data.actions),
+                    batch_dims=tf.rank(tf.squeeze(data.actions)),
+                    axis=1,
+                )
+            )
+            elementwise_td_loss = self.loss_func(tf.expand_dims(td_target, -1), tf.expand_dims(old_val, -1))
+            td_loss = tf.math.reduce_mean(elementwise_td_loss * data.weights)  # PER
+
+        grads = tape.gradient(td_loss, self.model.network.trainable_variables)
+        self.optimizer.apply_gradients(zip(grads, self.model.network.trainable_variables))
+
+        log_data = {"td_loss": td_loss, "elementwise_td_loss": elementwise_td_loss, "q_value": tf.reduce_mean(old_val)}
+        return log_data
+
+    def sync_target(self) -> None:
+        self.model_t.network.set_weights(self.model.network.get_weights())
+
+
+class Agent:
+    def __init__(self, **kwargs) -> None:
+        self.kwargs = kwargs
+
+        self.alg = Algorithm(**self.kwargs)
+        self.sample_step = 0
+        self.learn_step = 0
+
+    def predict(self, obs: np.ndarray) -> np.ndarray:
+        obs_ts = tf.convert_to_tensor(obs)
+        act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+        act_np = act_ts.numpy()
+        return act_np
+
+    def sample(self, obs: np.ndarray) -> np.ndarray:
+        if random.random() < self._get_epsilon():
+            act_np = np.array([self.kwargs["act_space"].sample() for _ in range(self.kwargs["num_envs"])])
+        else:
+            obs_ts = tf.convert_to_tensor(obs)
+            act_ts = tf.math.argmax(self.alg.predict(obs_ts), axis=1)
+            act_np = act_ts.numpy()
+
+        self.sample_step += self.kwargs["num_envs"]
+        if self.sample_step % self.kwargs["target_network_frequency"] == 0:
+            self.alg.sync_target()
+        return act_np
+
+    def learn(self, data: PrioritizedReplayBuffer.Samples[np.ndarray]) -> dict[str, Any]:
+        data_ts = PrioritizedReplayBuffer.Samples[tf.Tensor](
+            **{
+                item[0]: tf.convert_to_tensor(item[1]) if isinstance(item[1], np.ndarray) else item[1]
+                for item in dataclasses.asdict(data).items()
+            }
+        )
+
+        log_data = self.alg.learn(data_ts)
+        self.learn_step += 1
+        log_data["epsilon"] = self._get_epsilon()
+        return log_data
+
+    def _get_epsilon(self) -> float:
+        slope = (self.kwargs["end_epsilon"] - self.kwargs["start_epsilon"]) * (
+            self.sample_step / (self.kwargs["exploration_fraction"] * self.kwargs["total_timesteps"])
+        ) + self.kwargs["start_epsilon"]
+        return max(slope, self.kwargs["end_epsilon"])
+
+
+class Trainer:
+    def __init__(
+        self,
+        exp_name: str | None = None,
+        seed: int = 1,
+        cuda: bool = True,
+        capture_video: bool = False,
+        env_id: str = "CartPole-v1",
+        num_envs: int = 1,
+        total_timesteps: int = 500_000,
+        gamma: float = 0.99,
+        # Collect
+        alpha: float = 0.2,
+        beta: float = 0.6,
+        buffer_size: int = 10_000,
+        start_epsilon: float = 1.0,
+        end_epsilon: float = 0.05,
+        exploration_fraction: float = 0.5,
+        # Learn
+        batch_size: int = 128,
+        learning_rate: float = 2.5e-4,
+        # Train
+        learning_starts: int = 10_000,
+        target_network_frequency: int = 500,
+        train_frequency: int = 10,
+    ) -> None:
+        self.kwargs = locals()
+        self.kwargs.pop("self")
+
+        if self.kwargs["exp_name"] is None:
+            self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
+        self.kwargs["target_network_frequency"] = max(
+            self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
+        )
+        if not self.kwargs["cuda"]:
+            tf.config.experimental.set_visible_devices([], "GPU")
+
+        self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
+        assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
+
+        self.kwargs["obs_space"] = self.envs.single_observation_space
+        self.kwargs["act_space"] = self.envs.single_action_space
+
+        self.buffer = PrioritizedReplayBuffer(
+            self.kwargs["obs_space"],
+            self.kwargs["act_space"],
+            buffer_size=self.kwargs["buffer_size"],
+            alpha=self.kwargs["alpha"],
+        )
+
+        self.obs, _ = self.envs.reset(seed=self.kwargs["seed"])
+        self.agent = Agent(**self.kwargs)
+
+    def __call__(self) -> Generator[dict[str, Any], None, None]:
+        for _ in range(self.kwargs["learning_starts"]):
+            yield self._run_collect()
+        while self.agent.sample_step < self.kwargs["total_timesteps"]:
+            for _ in range(self.kwargs["train_frequency"]):
+                if not self.agent.sample_step < self.kwargs["total_timesteps"]:
+                    break
+                yield self._run_collect()
+            yield self._run_train()
+
+        self.envs.close_extras()
+
+    def _run_collect(self) -> dict[str, Any]:
+        act = self.agent.sample(self.obs)
+        next_obs, reward, terminated, truncated, infos = self.envs.step(act)
+
+        real_next_obs = next_obs.copy()
+        if "final_observation" in infos.keys():
+            for idx, final_obs in enumerate(infos["final_observation"]):
+                real_next_obs[idx] = real_next_obs[idx] if final_obs is None else final_obs
+
+        self.buffer.add(self.obs, real_next_obs, act, reward, terminated, infos)
+        self.obs = next_obs
+
+        if "final_info" in infos.keys():
+            final_info = next(item for item in infos["final_info"] if item is not None)
+            return {
+                "log_type": "collect",
+                "sample_step": self.agent.sample_step,
+                "logs": {
+                    "episodic_length": final_info["episode"]["l"][0],
+                    "episodic_return": final_info["episode"]["r"][0],
+                },
+            }
+        return {"log_type": "collect", "sample_step": self.agent.sample_step}
+
+    def _run_train(self) -> dict[str, Any]:
+        data = self.buffer.sample(batch_size=self.kwargs["batch_size"])
+        log_data = self.agent.learn(data)
+
+        loss_for_prior = log_data["elementwise_td_loss"].numpy() + 1e-6
+        self.buffer.update_priorities(data.indices, loss_for_prior)
+        log_data.pop("elementwise_td_loss")
+
+        return {"log_type": "train", "sample_step": self.agent.sample_step, "logs": log_data}
+
+    def _make_env(self, idx: int) -> Callable[[], gym.Env]:
+        def thunk() -> gym.Env:
+            env = gym.make(self.kwargs["env_id"], render_mode="rgb_array")
+            env = gym.wrappers.RecordEpisodeStatistics(env)
+            if self.kwargs["capture_video"]:
+                if idx == 0:
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
+            env.action_space.seed(self.kwargs["seed"] + idx)
+            env.observation_space.seed(self.kwargs["seed"] + idx)
+            return env
+
+        return thunk
+
+
+def wrapper_logger_tf(
+    wrapped: Callable[..., Generator[dict[str, Any], None, None]]
+) -> Callable[..., Generator[dict[str, Any], None, None]]:
+    import wandb
+
+    def setup_video_monitor() -> None:
+        vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
+        vcr.close_ = vcr.close  # type: ignore[attr-defined]
+
+        def close(self):
+            vcr.close_(self)
+            if self.path:
+                wandb.log({"videos": wandb.Video(self.path)})
+                self.path = None
+
+        vcr.close = close  # type: ignore[assignment]
+
+    @combine_signatures(wrapped)
+    def _wrapper(
+        *args,
+        track: bool = False,
+        wandb_project_name: str = "abcdrl",
+        wandb_tags: list[str] = [],
+        wandb_entity: str | None = None,
+        **kwargs,
+    ) -> Generator[dict[str, Any], None, None]:
+        instance = args[0]
+        if track:
+            wandb.init(
+                project=wandb_project_name,
+                tags=wandb_tags,
+                entity=wandb_entity,
+                sync_tensorboard=True,
+                config=instance.kwargs,
+                name=instance.kwargs["run_name"],
+                save_code=True,
+            )
+            setup_video_monitor()
+
+        writer = tf.summary.create_file_writer(f"runs/{instance.kwargs['run_name']}")
+        with writer.as_default():
+            tf.summary.text(
+                "hyperparameters",
+                "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
+                0,
+            )
+
+            gen = wrapped(*args, **kwargs)
+            for log_data in gen:
+                if "logs" in log_data:
+                    for log_item in log_data["logs"].items():
+                        tf.summary.scalar(f"{log_data['log_type']}/{log_item[0]}", log_item[1], log_data["sample_step"])
+                yield log_data
+
+    return _wrapper
+
+
+if __name__ == "__main__":
+    SEED = 1234
+    os.environ["PYTHONHASHSEED"] = str(SEED)
+    os.environ["TF_DETERMINISTIC_OPS"] = "1"
+    os.environ["TF_CUDNN_DETERMINISTIC"] = "1"
+    random.seed(SEED)
+    np.random.seed(SEED)
+    tf.random.set_seed(SEED)
+
+    Trainer.__call__ = wrapper_logger_tf(Trainer.__call__)  # type: ignore[assignment]
+    fire.Fire(
+        Trainer,
+        serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
+    )
diff --git a/abcdrl/pdqn.py b/abcdrl/pdqn_torch.py
similarity index 95%
rename from abcdrl/pdqn.py
rename to abcdrl/pdqn_torch.py
index 00129a3..065a6e4 100644
--- a/abcdrl/pdqn.py
+++ b/abcdrl/pdqn_torch.py
@@ -16,7 +16,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -126,8 +125,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -319,7 +318,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "CartPole-v1",
         num_envs: int = 1,
@@ -345,11 +344,11 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["target_network_frequency"] = max(
             self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
@@ -419,7 +418,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env.action_space.seed(self.kwargs["seed"] + idx)
             env.observation_space.seed(self.kwargs["seed"] + idx)
             return env
@@ -427,10 +426,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -454,7 +454,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -462,12 +461,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -484,15 +483,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/ppo.py b/abcdrl/ppo_torch.py
similarity index 96%
rename from abcdrl/ppo.py
rename to abcdrl/ppo_torch.py
index eb9daf7..77da173 100644
--- a/abcdrl/ppo.py
+++ b/abcdrl/ppo_torch.py
@@ -14,7 +14,6 @@
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
 from torch.distributions.normal import Normal
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -353,7 +352,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "Hopper-v4",
         num_envs: int = 1,
@@ -381,10 +380,10 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["batch_size"] = self.kwargs["num_envs"] * self.kwargs["num_steps"]
         self.kwargs["minibatch_size"] = self.kwargs["batch_size"] // self.kwargs["num_minibatches"]
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Box)
@@ -466,7 +465,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env = gym.wrappers.ClipAction(env)
             env = gym.wrappers.NormalizeObservation(env)
             env = gym.wrappers.TransformObservation(env, lambda obs: np.clip(obs, -10, 10))
@@ -479,10 +478,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -506,7 +506,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -514,12 +513,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -536,15 +535,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/sac.py b/abcdrl/sac_torch.py
similarity index 95%
rename from abcdrl/sac.py
rename to abcdrl/sac_torch.py
index 7c137cd..70566de 100644
--- a/abcdrl/sac.py
+++ b/abcdrl/sac_torch.py
@@ -15,7 +15,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -65,8 +64,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -305,7 +304,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "Hopper-v4",
         num_envs: int = 1,
@@ -331,14 +330,14 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["policy_frequency"] = max(
             self.kwargs["policy_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
         self.kwargs["target_network_frequency"] = max(
             self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Box)
@@ -404,7 +403,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env.action_space.seed(self.kwargs["seed"] + idx)
             env.observation_space.seed(self.kwargs["seed"] + idx)
             return env
@@ -412,10 +411,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -439,7 +439,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -447,12 +446,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -469,15 +468,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/td3.py b/abcdrl/td3_torch.py
similarity index 95%
rename from abcdrl/td3.py
rename to abcdrl/td3_torch.py
index 0708ef2..a7648f6 100644
--- a/abcdrl/td3.py
+++ b/abcdrl/td3_torch.py
@@ -15,7 +15,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -65,8 +64,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -278,7 +277,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "Hopper-v4",
         num_envs: int = 1,
@@ -303,11 +302,11 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["policy_frequency"] = max(
             self.kwargs["policy_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Box)
@@ -373,7 +372,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env.action_space.seed(self.kwargs["seed"] + idx)
             env.observation_space.seed(self.kwargs["seed"] + idx)
             return env
@@ -381,10 +380,11 @@ def thunk() -> gym.Env:
         return thunk
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -408,7 +408,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -416,12 +415,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -438,15 +437,16 @@ def _wrapper(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
         Trainer,
         serialize=lambda gen: (log_data for log_data in gen if "logs" in log_data and log_data["log_type"] != "train"),
diff --git a/abcdrl/utils/__init__.py b/abcdrl/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/abcdrl_copy_from/dqn_all_wrappers.py b/abcdrl/utils/dqn_all_wrappers_torch.py
similarity index 95%
rename from abcdrl_copy_from/dqn_all_wrappers.py
rename to abcdrl/utils/dqn_all_wrappers_torch.py
index 7fa2abc..e57dc1e 100644
--- a/abcdrl_copy_from/dqn_all_wrappers.py
+++ b/abcdrl/utils/dqn_all_wrappers_torch.py
@@ -15,7 +15,6 @@
 import torch.nn.functional as F
 import torch.optim as optim
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 SamplesItemType = TypeVar("SamplesItemType", torch.Tensor, np.ndarray)
 
@@ -65,8 +64,8 @@ def add(
         obs: np.ndarray,
         next_obs: np.ndarray,
         act: np.ndarray,
-        rew: float,
-        done: bool,
+        rew: np.ndarray,
+        done: np.ndarray,
         infos: dict[str, Any],
     ) -> None:
         for obs_i, next_obs_i, act_i, rew_i, done_i in zip(obs, next_obs, act, rew, done):  # type: ignore[call-overload]
@@ -208,7 +207,7 @@ def __init__(
         self,
         exp_name: str | None = None,
         seed: int = 1,
-        device: str | torch.device = "auto",
+        cuda: bool = True,
         capture_video: bool = False,
         env_id: str = "CartPole-v1",
         num_envs: int = 1,
@@ -232,11 +231,11 @@ def __init__(
 
         if self.kwargs["exp_name"] is None:
             self.kwargs["exp_name"] = f"{self.kwargs['env_id']}__{os.path.basename(__file__).rstrip('.py')}"
+        self.kwargs["run_name"] = f"{self.kwargs['exp_name']}__{self.kwargs['seed']}__{int(time.time())}"
         self.kwargs["target_network_frequency"] = max(
             self.kwargs["target_network_frequency"] // self.kwargs["num_envs"] * self.kwargs["num_envs"], 1
         )
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.envs = gym.vector.SyncVectorEnv([self._make_env(i) for i in range(self.kwargs["num_envs"])])  # type: ignore[arg-type]
         assert isinstance(self.envs.single_action_space, gym.spaces.Discrete)
@@ -301,7 +300,7 @@ def thunk() -> gym.Env:
             env = gym.wrappers.RecordEpisodeStatistics(env)
             if self.kwargs["capture_video"]:
                 if idx == 0:
-                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['exp_name']}")
+                    env = gym.wrappers.RecordVideo(env, f"videos/{self.kwargs['run_name']}")
             env.action_space.seed(self.kwargs["seed"] + idx)
             env.observation_space.seed(self.kwargs["seed"] + idx)
             return env
@@ -348,10 +347,11 @@ def _wrapper(
     return _wrapper
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -375,7 +375,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -383,12 +382,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
@@ -441,16 +440,17 @@ def _wrapper(*args, **kwargs) -> Generator[dict[str, Any], None, None]:
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED = 1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
     Trainer.__call__ = wrapper_eval_step(Trainer.__call__)  # type: ignore[assignment]
-    Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
+    Trainer.__call__ = wrapper_logger_torch(Trainer.__call__)  # type: ignore[assignment]
     Trainer.__call__ = wrapper_save_model(Trainer.__call__)  # type: ignore[assignment]
     Trainer.__call__ = wrapper_print_filter(Trainer.__call__)  # type: ignore[assignment]
     fire.Fire(
diff --git a/abcdrl_copy_from/wrapper_eval_step.py b/abcdrl/utils/wrapper_eval_step.py
similarity index 100%
rename from abcdrl_copy_from/wrapper_eval_step.py
rename to abcdrl/utils/wrapper_eval_step.py
diff --git a/abcdrl/utils/wrapper_logger_tf.py b/abcdrl/utils/wrapper_logger_tf.py
new file mode 100644
index 0000000..79e4643
--- /dev/null
+++ b/abcdrl/utils/wrapper_logger_tf.py
@@ -0,0 +1,64 @@
+from __future__ import annotations
+
+from typing import Any, Callable, Generator
+
+import gymnasium as gym
+import tensorflow as tf
+from combine_signatures.combine_signatures import combine_signatures
+
+
+def wrapper_logger_tf(
+    wrapped: Callable[..., Generator[dict[str, Any], None, None]]
+) -> Callable[..., Generator[dict[str, Any], None, None]]:
+    import wandb
+
+    def setup_video_monitor() -> None:
+        vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
+        vcr.close_ = vcr.close  # type: ignore[attr-defined]
+
+        def close(self):
+            vcr.close_(self)
+            if self.path:
+                wandb.log({"videos": wandb.Video(self.path)})
+                self.path = None
+
+        vcr.close = close  # type: ignore[assignment]
+
+    @combine_signatures(wrapped)
+    def _wrapper(
+        *args,
+        track: bool = False,
+        wandb_project_name: str = "abcdrl",
+        wandb_tags: list[str] = [],
+        wandb_entity: str | None = None,
+        **kwargs,
+    ) -> Generator[dict[str, Any], None, None]:
+        instance = args[0]
+        if track:
+            wandb.init(
+                project=wandb_project_name,
+                tags=wandb_tags,
+                entity=wandb_entity,
+                sync_tensorboard=True,
+                config=instance.kwargs,
+                name=instance.kwargs["run_name"],
+                save_code=True,
+            )
+            setup_video_monitor()
+
+        writer = tf.summary.create_file_writer(f"runs/{instance.kwargs['run_name']}")
+        with writer.as_default():
+            tf.summary.text(
+                "hyperparameters",
+                "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
+                0,
+            )
+
+            gen = wrapped(*args, **kwargs)
+            for log_data in gen:
+                if "logs" in log_data:
+                    for log_item in log_data["logs"].items():
+                        tf.summary.scalar(f"{log_data['log_type']}/{log_item[0]}", log_item[1], log_data["sample_step"])
+                yield log_data
+
+    return _wrapper
diff --git a/abcdrl_copy_from/wrapper_logger.py b/abcdrl/utils/wrapper_logger_torch.py
similarity index 87%
rename from abcdrl_copy_from/wrapper_logger.py
rename to abcdrl/utils/wrapper_logger_torch.py
index 0f3fea3..087c302 100644
--- a/abcdrl_copy_from/wrapper_logger.py
+++ b/abcdrl/utils/wrapper_logger_torch.py
@@ -1,17 +1,16 @@
 from __future__ import annotations
 
-import time
 from typing import Any, Callable, Generator
 
 import gymnasium as gym
 from combine_signatures.combine_signatures import combine_signatures
-from torch.utils.tensorboard import SummaryWriter
 
 
-def wrapper_logger(
+def wrapper_logger_torch(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
     import wandb
+    from torch.utils.tensorboard import SummaryWriter
 
     def setup_video_monitor() -> None:
         vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -35,7 +34,6 @@ def _wrapper(
         **kwargs,
     ) -> Generator[dict[str, Any], None, None]:
         instance = args[0]
-        exp_name_ = f"{instance.kwargs['exp_name']}__{instance.kwargs['seed']}__{int(time.time())}"
         if track:
             wandb.init(
                 project=wandb_project_name,
@@ -43,12 +41,12 @@ def _wrapper(
                 entity=wandb_entity,
                 sync_tensorboard=True,
                 config=instance.kwargs,
-                name=exp_name_,
+                name=instance.kwargs["run_name"],
                 save_code=True,
             )
             setup_video_monitor()
 
-        writer = SummaryWriter(f"runs/{exp_name_}")
+        writer = SummaryWriter(f"runs/{instance.kwargs['run_name']}")
         writer.add_text(
             "hyperparameters",
             "|param|value|\n|-|-|\n" + "\n".join([f"|{key}|{value}|" for key, value in instance.kwargs.items()]),
diff --git a/abcdrl_copy_from/wrapper_print_filter.py b/abcdrl/utils/wrapper_print_filter.py
similarity index 100%
rename from abcdrl_copy_from/wrapper_print_filter.py
rename to abcdrl/utils/wrapper_print_filter.py
diff --git a/abcdrl_copy_from/wrapper_save_model.py b/abcdrl/utils/wrapper_save_model.py
similarity index 100%
rename from abcdrl_copy_from/wrapper_save_model.py
rename to abcdrl/utils/wrapper_save_model.py
diff --git a/abcdrl_copy_from/__init__.py b/abcdrl_copy_from/__init__.py
deleted file mode 100644
index c0eb10d..0000000
--- a/abcdrl_copy_from/__init__.py
+++ /dev/null
@@ -1,6 +0,0 @@
-import abcdrl_copy_from.dqn_all_wrappers  # noqa
-
-from .wrapper_eval_step import wrapper_eval_step  # noqa
-from .wrapper_logger import wrapper_logger  # noqa
-from .wrapper_print_filter import wrapper_print_filter  # noqa
-from .wrapper_save_model import wrapper_save_model  # noqa
diff --git a/benchmark.py b/benchmark.py
index 1e80628..b60722c 100644
--- a/benchmark.py
+++ b/benchmark.py
@@ -9,22 +9,24 @@
 import tomli
 
 
-def train_process(alg: str, env_id: str, kwargs: dict[str, Any]) -> None:
+def train_process(alg: str, framework: str, env_id: str, kwargs: dict[str, Any]) -> None:
     with open("benchmark.toml", "rb") as file:
-        cmd = tomli.load(file)[alg][env_id]
+        cmd = tomli.load(file)[alg][framework][env_id]
     for param in kwargs.items():
         if isinstance(param[1], list):
             cmd += f' --{param[0]} "{param[1]}"'
         else:
             cmd += f" --{param[0]} {param[1]}"
+    print(cmd)
     subprocess.run(cmd, shell=True, check=True)
 
 
 def main(
     algs: str | list[str] = ["dqn"],
+    frameworks: str | list[str] = ["torch"],
     env_ids: str | list[str] = ["CartPole-v1"],
     seeds: int | list[int] = [1],
-    devices: str | list[str] = ["auto"],
+    cuda: bool = True,
     workers: int = 3,
     track: bool = False,
     wandb_project_name: str = "abcdrl",
@@ -34,12 +36,12 @@ def main(
 ):
     if not isinstance(algs, list):
         algs = [algs]
+    if not isinstance(frameworks, list):
+        frameworks = [frameworks]
     if not isinstance(env_ids, list):
         env_ids = [env_ids]
     if not isinstance(seeds, list):
         seeds = [seeds]
-    if not isinstance(devices, list):
-        devices = [devices]
     if not isinstance(wandb_tags, list):
         wandb_tags = [wandb_tags]
 
@@ -47,9 +49,9 @@ def main(
     wandb_tags.append(git_commit_sha)
 
     with ThreadPoolExecutor(max_workers=workers) as executor:
-        for (alg, env_id, seed), device in zip(itertools.product(algs, env_ids, seeds), itertools.cycle(devices)):
+        for alg, framework, env_id, seed in itertools.product(algs, frameworks, env_ids, seeds):
             kwargs = {
-                "device": device,
+                "cuda": cuda,
                 "track": track,
                 "wandb-project-name": wandb_project_name,
                 "wandb-entity": wandb_entity,
@@ -57,7 +59,7 @@ def main(
                 "capture-video": capture_video,
                 "seed": seed,
             }
-            executor.submit(train_process, alg, env_id, kwargs)
+            executor.submit(train_process, alg, framework, env_id, kwargs)
 
 
 if __name__ == "__main__":
diff --git a/benchmark.toml b/benchmark.toml
index 82c9f67..91c8f1b 100644
--- a/benchmark.toml
+++ b/benchmark.toml
@@ -1,7 +1,9 @@
 [dqn]
 
+[dqn.torch]
+
 CartPole-v1 = """
-    python abcdrl/dqn.py \
+    python abcdrl/dqn_torch.py \
         --env-id CartPole-v1 \
         --num-envs 1 \
         --total-timesteps 500000 \
@@ -18,7 +20,7 @@ CartPole-v1 = """
 """
 
 BreakoutNoFrameskip-v4 = """
-    python abcdrl/dqn_atari.py \
+    python abcdrl/dqn_atari_torch.py \
         --env-id BreakoutNoFrameskip-v4 \
         --num-envs 1 \
         --total-timesteps 10000000 \
@@ -35,7 +37,7 @@ BreakoutNoFrameskip-v4 = """
 """
 
 PongNoFrameskip-v4 = """
-    python abcdrl/dqn_atari.py \
+    python abcdrl/dqn_atari_torch.py \
         --env-id PongNoFrameskip-v4 \
         --num-envs 1 \
         --total-timesteps 10000000 \
@@ -52,7 +54,7 @@ PongNoFrameskip-v4 = """
 """
 
 BeamRiderNoFrameskip-v4 = """
-    python abcdrl/dqn_atari.py \
+    python abcdrl/dqn_atari_torch.py \
         --env-id BeamRiderNoFrameskip-v4 \
         --num-envs 1 \
         --total-timesteps 10000000 \
@@ -68,11 +70,83 @@ BeamRiderNoFrameskip-v4 = """
         --train-frequency 4 \
 """
 
+[dqn.tf]
+
+CartPole-v1 = """
+    python abcdrl/dqn_tf.py \
+        --env-id CartPole-v1 \
+        --num-envs 1 \
+        --total-timesteps 500000 \
+        --gamma 0.99 \
+        --buffer-size 10000 \
+        --start-epsilon 1.0 \
+        --end-epsilon 0.05 \
+        --exploration-fraction 0.5 \
+        --batch-size 128 \
+        --learning-rate 2.5e-4 \
+        --learning-starts 10000 \
+        --target-network-frequency 500 \
+        --train-frequency 10 \
+"""
+
+BreakoutNoFrameskip-v4 = """
+    python abcdrl/dqn_atari_tf.py \
+        --env-id BreakoutNoFrameskip-v4 \
+        --num-envs 1 \
+        --total-timesteps 10000000 \
+        --gamma 0.99 \
+        --buffer-size 1000000 \
+        --start-epsilon 1.0 \
+        --end-epsilon 0.01 \
+        --exploration-fraction 0.1 \
+        --batch-size 32 \
+        --learning-rate 1e-4 \
+        --learning-starts 50000 \
+        --target-network-frequency 1000 \
+        --train-frequency 4 \
+"""
+
+PongNoFrameskip-v4 = """
+    python abcdrl/dqn_atari_tf.py \
+        --env-id PongNoFrameskip-v4 \
+        --num-envs 1 \
+        --total-timesteps 10000000 \
+        --gamma 0.99 \
+        --buffer-size 1000000 \
+        --start-epsilon 1.0 \
+        --end-epsilon 0.01 \
+        --exploration-fraction 0.1 \
+        --batch-size 32 \
+        --learning-rate 1e-4 \
+        --learning-starts 50000 \
+        --target-network-frequency 1000 \
+        --train-frequency 4 \
+"""
+
+BeamRiderNoFrameskip-v4 = """
+    python abcdrl/dqn_atari_tf.py \
+        --env-id BeamRiderNoFrameskip-v4 \
+        --num-envs 1 \
+        --total-timesteps 10000000 \
+        --gamma 0.99 \
+        --buffer-size 1000000 \
+        --start-epsilon 1.0 \
+        --end-epsilon 0.01 \
+        --exploration-fraction 0.1 \
+        --batch-size 32 \
+        --learning-rate 1e-4 \
+        --learning-starts 50000 \
+        --target-network-frequency 1000 \
+        --train-frequency 4 \
+"""
 
 [ddqn]
 
+
+[ddqn.torch]
+
 CartPole-v1 = """
-    python abcdrl/ddqn.py \
+    python abcdrl/ddqn_torch.py \
         --env-id CartPole-v1 \
         --num-envs 1 \
         --total-timesteps 500000 \
@@ -88,11 +162,54 @@ CartPole-v1 = """
         --train-frequency 10 \
 """
 
+[ddqn.tf]
+
+CartPole-v1 = """
+    python abcdrl/ddqn_tf.py \
+        --env-id CartPole-v1 \
+        --num-envs 1 \
+        --total-timesteps 500000 \
+        --gamma 0.99 \
+        --buffer-size 10000 \
+        --start-epsilon 1.0 \
+        --end-epsilon 0.05 \
+        --exploration-fraction 0.5 \
+        --batch-size 128 \
+        --learning-rate 2.5e-4 \
+        --learning-starts 10000 \
+        --target-network-frequency 500 \
+        --train-frequency 10 \
+"""
 
 [pdqn]
 
+
+[pdqn.torch]
+
 CartPole-v1 = """
-    python abcdrl/pdqn.py \
+    python abcdrl/pdqn_torch.py \
+        --env-id CartPole-v1 \
+        --num-envs 1 \
+        --total-timesteps 500000 \
+        --gamma 0.99 \
+        --alpha 0.2 \
+        --beta 0.6 \
+        --buffer-size 10000 \
+        --start-epsilon 1.0 \
+        --end-epsilon 0.05 \
+        --exploration-fraction 0.5 \
+        --batch-size 128 \
+        --learning-rate 2.5e-4 \
+        --learning-starts 10000 \
+        --target-network-frequency 500 \
+        --train-frequency 10 \
+"""
+
+
+[pdqn.tf]
+
+CartPole-v1 = """
+    python abcdrl/pdqn_tf.py \
         --env-id CartPole-v1 \
         --num-envs 1 \
         --total-timesteps 500000 \
@@ -113,8 +230,11 @@ CartPole-v1 = """
 
 [ddpg]
 
+
+[ddpg.torch]
+
 Pendulum-v1 = """
-    python abcdrl/ddpg.py \
+    python abcdrl/ddpg_torch.py \
         --env-id Pendulum-v1 \
         --num-envs 1 \
         --total-timesteps 50000 \
@@ -130,7 +250,7 @@ Pendulum-v1 = """
 """
 
 Hopper-v4 = """
-    python abcdrl/ddpg.py \
+    python abcdrl/ddpg_torch.py \
         --env-id Hopper-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -146,7 +266,7 @@ Hopper-v4 = """
 """
 
 Walker2d-v4 = """
-    python abcdrl/ddpg.py \
+    python abcdrl/ddpg_torch.py \
         --env-id Walker2d-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -162,7 +282,7 @@ Walker2d-v4 = """
 """
 
 HalfCheetah-v4 = """
-    python abcdrl/ddpg.py \
+    python abcdrl/ddpg_torch.py \
         --env-id HalfCheetah-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -180,8 +300,11 @@ HalfCheetah-v4 = """
 
 [td3]
 
+
+[td3.torch]
+
 Pendulum-v1 = """
-    python abcdrl/td3.py \
+    python abcdrl/td3_torch.py \
         --env-id Pendulum-v1 \
         --num-envs 1 \
         --total-timesteps 50000 \
@@ -199,7 +322,7 @@ Pendulum-v1 = """
 """
 
 Hopper-v4 = """
-    python abcdrl/td3.py \
+    python abcdrl/td3_torch.py \
         --env-id Hopper-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -217,7 +340,7 @@ Hopper-v4 = """
 """
 
 Walker2d-v4 = """
-    python abcdrl/td3.py \
+    python abcdrl/td3_torch.py \
         --env-id Walker2d-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -235,7 +358,7 @@ Walker2d-v4 = """
 """
 
 HalfCheetah-v4 = """
-    python abcdrl/td3.py \
+    python abcdrl/td3_torch.py \
         --env-id HalfCheetah-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -255,8 +378,10 @@ HalfCheetah-v4 = """
 
 [sac]
 
+[sac.torch]
+
 Pendulum-v1 = """
-    python abcdrl/sac.py \
+    python abcdrl/sac_torch.py \
         --env-id Pendulum-v1 \
         --num-envs 1 \
         --total-timesteps 50000 \
@@ -275,7 +400,7 @@ Pendulum-v1 = """
 """
 
 Hopper-v4 = """
-    python abcdrl/sac.py \
+    python abcdrl/sac_torch.py \
         --env-id Hopper-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -294,7 +419,7 @@ Hopper-v4 = """
 """
 
 Walker2d-v4 = """
-    python abcdrl/sac.py \
+    python abcdrl/sac_torch.py \
         --env-id Walker2d-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -313,7 +438,7 @@ Walker2d-v4 = """
 """
 
 HalfCheetah-v4 = """
-    python abcdrl/sac.py \
+    python abcdrl/sac_torch.py \
         --env-id HalfCheetah-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -334,8 +459,10 @@ HalfCheetah-v4 = """
 
 [ppo]
 
+[ppo.torch]
+
 Pendulum-v1 = """
-    python abcdrl/ppo.py \
+    python abcdrl/ppo_torch.py \
         --env-id Pendulum-v1 \
         --num-envs 1 \
         --total-timesteps 50000 \
@@ -355,7 +482,7 @@ Pendulum-v1 = """
 """
 
 Hopper-v4 = """
-    python abcdrl/ppo.py \
+    python abcdrl/ppo_torch.py \
         --env-id Hopper-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -375,7 +502,7 @@ Hopper-v4 = """
 """
 
 Walker2d-v4 = """
-    python abcdrl/ppo.py \
+    python abcdrl/ppo_torch.py \
         --env-id Walker2d-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
@@ -395,7 +522,7 @@ Walker2d-v4 = """
 """
 
 HalfCheetah-v4 = """
-    python abcdrl/ppo.py \
+    python abcdrl/ppo_torch.py \
         --env-id HalfCheetah-v4 \
         --num-envs 1 \
         --total-timesteps 1000000 \
diff --git a/docs/features.en.md b/docs/features.en.md
index ca159e0..7682c6a 100644
--- a/docs/features.en.md
+++ b/docs/features.en.md
@@ -11,11 +11,15 @@
     In order to facilitate the comparison of the differences between algorithms and to unify the code style, the code will try to minimize the differences between the wired code files as shown in the diagram below.
     ``` mermaid
     graph LR
-    A[dqn.py] -->B[ddpg.py];
-    B -->C[td3.py];
-    C -->D[sac.py];
-    B -->E[ppo.py];
-    A -->F[ddqn.py];
-    A -->G[pdqn.py];
-    A -->H[dqn_atari.py];
+    A[dqn_torch.py] -->B[ddpg_torch.py];
+    B -->C[td3_torch.py];
+    C -->D[sac_torch.py];
+    B -->E[ppo_torch.py];
+    A -->F[ddqn_torch.py];
+    A -->G[pdqn_torch.py];
+    A -->H[dqn_atari_torch.py];
+    A -->I[dqn_tf.py];
+    F -->J[ddqn_tf.py];
+    G -->K[pdqn_tf.py];
+    H -->L[dqn_atari_tf.py];
     ```
diff --git a/docs/features.zh.md b/docs/features.zh.md
index ac98ab7..517c2cd 100644
--- a/docs/features.zh.md
+++ b/docs/features.zh.md
@@ -11,11 +11,15 @@
     为了便于比较不同算法之间的差异和统一代码风格，我们的代码将按照下述的关系图，尽力做到连线的代码文件差异的最小化。
     ``` mermaid
     graph LR
-    A[dqn.py] -->B[ddpg.py];
-    B -->C[td3.py];
-    C -->D[sac.py];
-    B -->E[ppo.py];
-    A -->F[ddqn.py];
-    A -->G[pdqn.py];
-    A -->H[dqn_atari.py];
+    A[dqn_torch.py] -->B[ddpg_torch.py];
+    B -->C[td3_torch.py];
+    C -->D[sac_torch.py];
+    B -->E[ppo_torch.py];
+    A -->F[ddqn_torch.py];
+    A -->G[pdqn_torch.py];
+    A -->H[dqn_atari_torch.py];
+    A -->I[dqn_tf.py];
+    F -->J[ddqn_tf.py];
+    G -->K[pdqn_tf.py];
+    H -->L[dqn_atari_tf.py];
     ```
diff --git a/docs/index.en.md b/docs/index.en.md
index 410cef3..b944145 100644
--- a/docs/index.en.md
+++ b/docs/index.en.md
@@ -32,16 +32,16 @@ abcdRL is a **Modular Single-file Reinforcement Learning Algorithms Library** th
 
 ***Weights & Biases Benchmark Report ➡️ [report.abcdrl.xyz](https://report.abcdrl.xyz)***
 
-- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236)
-- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971)
-- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477)
-- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290)
-- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477)
+- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236) <sub>`dqn_torch.py`, `dqn_tf.py`, `dqn_atari_torch.py`, `dqn_atari_tf.py`</sub>
+- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971) <sub>`ddpg_torch.py`</sub>
+- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477) <sub>`td3_torch.py`</sub>
+- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290) <sub>`sac_torch.py`</sub>
+- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477) <sub>`ppo_torch.py`</sub>
 
 ---
 
-- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461)
-- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952)
+- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461) <sub>`ddqn_torch.py`, `ddqn_tf.py`</sub>
+- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952) <sub>`pdqn_torch.py`, `pdqn_tf.py`</sub>
 
 ## Citing abcdRL
 
diff --git a/docs/index.zh.md b/docs/index.zh.md
index 3f64c2b..a45cf9a 100644
--- a/docs/index.zh.md
+++ b/docs/index.zh.md
@@ -17,7 +17,7 @@ abcdRL 是一个**模块化单文件强化学习代码库**，提供“有但不
 *阅读代码时，在单文件代码中，快速了解算法的完整实现细节；改进算法时，得益于轻量的模块化设计，只需专注于少量的模块。*
 
 !!! quote "Ref"
-    abcdRL 主要参考了 [vwxyzjn/cleanRL](https://github.com/vwxyzjn/cleanrl/) 的单文件设计哲学和 [PaddlePaddle/PARL](https://github.com/PaddlePaddle/PARL/) 的模块设计。
+    abcdRL 主要参考了 [vwxyzjn/cleanrl](https://github.com/vwxyzjn/cleanrl/) 的单文件设计哲学和 [PaddlePaddle/PARL](https://github.com/PaddlePaddle/PARL/) 的模块设计。
 
 ***路线图🗺️ [#57](https://github.com/sdpkjc/abcdrl/issues/57)***
 
@@ -32,16 +32,16 @@ abcdRL 是一个**模块化单文件强化学习代码库**，提供“有但不
 
 ***Weights & Biases 性能报告 ➡️ [report.abcdrl.xyz](https://report.abcdrl.xyz)***
 
-- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236)
-- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971)
-- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477)
-- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290)
-- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477)
+- [Deep Q Network (DQN)](https://doi.org/10.1038/nature14236) <sub>`dqn_torch.py`, `dqn_tf.py`, `dqn_atari_torch.py`, `dqn_atari_tf.py`</sub>
+- [Deep Deterministic Policy Gradient (DDPG)](http://arxiv.org/abs/1509.02971) <sub>`ddpg_torch.py`</sub>
+- [Twin Delayed Deep Deterministic Policy Gradient (TD3)](http://arxiv.org/abs/1802.09477) <sub>`td3_torch.py`</sub>
+- [Soft Actor-Critic (SAC)](http://arxiv.org/abs/1801.01290) <sub>`sac_torch.py`</sub>
+- [Proximal Policy Optimization (PPO)](http://arxiv.org/abs/1802.09477) <sub>`ppo_torch.py`</sub>
 
 ---
 
-- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461)
-- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952)
+- [Double Deep Q Network (DDQN)](http://arxiv.org/abs/1509.06461) <sub>`ddqn_torch.py`, `ddqn_tf.py`</sub>
+- [Prioritized Deep Q Network (PDQN)](http://arxiv.org/abs/1511.05952) <sub>`pdqn_torch.py`, `pdqn_tf.py`</sub>
 
 ## 引用 abcdRL
 
diff --git a/docs/install.en.md b/docs/install.en.md
index 0e6ec0b..46dab40 100644
--- a/docs/install.en.md
+++ b/docs/install.en.md
@@ -12,7 +12,7 @@
     ```shell
     # 0. Prerequisites: Docker
     # 1. Run DQN algorithm
-    docker run --rm sdpkjc/abcdrl python abcdrl/dqn.py
+    docker run --rm sdpkjc/abcdrl python abcdrl/dqn_torch.py
     ```
 
 === "GPU"
@@ -20,7 +20,7 @@
     ```shell
     # 0. Prerequisites: Docker & Nvidia Drive & NVIDIA Container Toolkit
     # 1. Run DQN algorithm
-    docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn.py
+    docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn_torch.py
     ```
 
     !!! note
@@ -44,10 +44,10 @@
     # 2. Install dependencies
     pip install -r requirements/requirements.txt
     # 3. Run DQN algorithm
-    python abcdrl/dqn.py
+    python abcdrl/dqn_torch.py
     ```
 
-=== "GPU"
+=== "GPU with PyTorch"
 
     ```shell
     # 0. Prerequisites: Conda & Nvidia Driver
@@ -58,12 +58,32 @@
     # 3. Install cudatoolkit and the corresponding version of Pytorch
     conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge
     # 4. Install dependencies
-    pip install -r requirements/requirements.txt
+    pip install -r requirements/requirements-torch.txt
     # 5. Run DQN algorithm
-    python abcdrl/dqn.py
+    python abcdrl/dqn_torch.py
     ```
 
     !!! note
         There are many ways to install pytorch, refer to [Mu Li's video tutorials](https://b23.tv/qvAxVzd) for details.
 
         Version selection of `cudatoolkit` is related to Nvidia Driver version, refer to [Mu Li's video tutorial](https://b23.tv/qvAxVzd) and [Pytorch installation page](https://pytorch.org/get-started/locally/).
+
+=== "GPU with TensorFlow2"
+
+    ```shell
+    # 0. Prerequisites: Conda & Nvidia Driver
+    # 1. Pull git repository from github
+    git clone https://github.com/sdpkjc/abcdrl.git && cd abcdrl
+    # 2. Create and activate virtual environment
+    conda create -n abcdrl python=3.9 pip && conda activate abcdrl
+    # 3. Install cudatoolkit
+    conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0
+    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
+    # 4. Install dependencies
+    pip install -r requirements/requirements-tf.txt
+    # 5. Run DQN algorithm
+    python abcdrl/dqn_tf.py
+    ```
+
+    !!! quote "Ref"
+        [TensorFlow Official Guide](https://www.tensorflow.org/install/pip)
diff --git a/docs/install.zh.md b/docs/install.zh.md
index 70a1cc2..559522a 100644
--- a/docs/install.zh.md
+++ b/docs/install.zh.md
@@ -12,7 +12,7 @@
     ```shell
     # 0. 安装 Docker
     # 1. 运行 DQN 算法
-    docker run --rm sdpkjc/abcdrl python abcdrl/dqn.py
+    docker run --rm sdpkjc/abcdrl python abcdrl/dqn_torch.py
     ```
 
 === "GPU"
@@ -20,7 +20,7 @@
     ```shell
     # 0. 安装 Docker & Nvidia Drive & NVIDIA Container Toolkit
     # 1. 运行 DQN 算法
-    docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn.py
+    docker run --rm --gpus all sdpkjc/abcdrl python abcdrl/dqn_torch.py
     ```
 
     !!! note
@@ -44,10 +44,10 @@
     # 2. 安装依赖
     pip install -r requirements/requirements.txt
     # 3. 运行 DQN 算法
-    python abcdrl/dqn.py
+    python abcdrl/dqn_torch.py
     ```
 
-=== "GPU"
+=== "GPU with PyTorch"
 
     ```shell
     # 0. 安装 Conda & Nvidia Driver
@@ -60,10 +60,30 @@
     # 4. 安装依赖
     pip install -r requirements/requirements.txt
     # 5. 运行 DQN 算法
-    python abcdrl/dqn.py
+    python abcdrl/dqn_torch.py
     ```
 
     !!! note
         Pytorch 安装方法有多种可选，具体可参考视频 [李沐：环境安装，BERT、GPT、T5 性能测试，和横向对比【100亿模型计划】-哔哩哔哩](https://b23.tv/qvAxVzd) 。
 
         `cudatoolkit` 的版本选择与 Nvidia Driver 版本相关，请参考[视频教程](https://b23.tv/qvAxVzd)和 [Pytorch 官网安装页面](https://pytorch.org/get-started/locally/)。
+
+=== "GPU with TensorFlow2"
+
+    ```shell
+    # 0. 安装 Conda & Nvidia Driver
+    # 1. 拉取代码仓库
+    git clone https://github.com/sdpkjc/abcdrl.git && cd abcdrl
+    # 2. 建立虚拟环境并激活
+    conda create -n abcdrl python=3.9 pip && conda activate abcdrl
+    # 3. 安装 cudatoolkit
+    conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0
+    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
+    # 4. 安装依赖
+    pip install -r requirements/requirements-tf.txt
+    # 5. 运行 DQN 算法
+    python abcdrl/dqn_tf.py
+    ```
+
+    !!! quote "Ref"
+        [TensorFlow 官方指南](https://www.tensorflow.org/install/pip)
diff --git a/docs/modify.en.md b/docs/modify.en.md
index 45d7e19..76a403c 100644
--- a/docs/modify.en.md
+++ b/docs/modify.en.md
@@ -51,7 +51,7 @@ Our modular design does not prescribe a strict interface, and you are free to mo
 
 ### Writing Decorator
 
-The generic feature is implemented as a decorator, you can refer to the code below and `abcdrl_copy_from/wrapper_*.py` file to implement the new feature you want and apply it to all algorithms.
+The generic feature is implemented as a decorator, you can refer to the code below and `abcdrl/utils/wrapper_*.py` file to implement the new feature you want and apply it to all algorithms.
 
 ```python hl_lines="8-9 13 15"
 from combine_signatures.combine_signatures import combine_signatures
@@ -74,7 +74,7 @@ def wrapper_example(
 
 ### Using Decorator
 
-```python hl_lines="1-11 24-25"
+```python hl_lines="1-11 25-26"
 # Step 1：Copy the decorators you need
 def wrapper_example(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
@@ -89,13 +89,14 @@ def wrapper_example(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED=1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
     Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
     # Step 2：Decorate the Trainer.__call__ function
diff --git a/docs/modify.zh.md b/docs/modify.zh.md
index c59a965..f72fe3d 100644
--- a/docs/modify.zh.md
+++ b/docs/modify.zh.md
@@ -51,7 +51,7 @@
 
 ### 编写装饰器
 
-我们的通用功能主要通过装饰器实现，可以参考以下代码和 `abcdrl_copy_from/wrapper_*.py` 文件，实现你想要的新功能并应用到所有算法上。
+我们的通用功能主要通过装饰器实现，可以参考以下代码和 `abcdrl/utils/wrapper_*.py` 文件，实现你想要的新功能并应用到所有算法上。
 
 ```python hl_lines="8-9 13 15"
 from combine_signatures.combine_signatures import combine_signatures
@@ -74,7 +74,7 @@ def wrapper_example(
 
 ### 使用装饰器
 
-```python hl_lines="1-11 24-25"
+```python hl_lines="1-11 25-26"
 # 第一步：复制需要的装饰器
 def wrapper_example(
     wrapped: Callable[..., Generator[dict[str, Any], None, None]]
@@ -89,13 +89,14 @@ def wrapper_example(
 
 
 if __name__ == "__main__":
-    torch.manual_seed(1234)
-    torch.cuda.manual_seed(1234)
-    np.random.seed(1234)
-    random.seed(1234)
+    SEED=1234
+    random.seed(SEED)
+    np.random.seed(SEED)
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
     torch.backends.cudnn.benchmark = False
     torch.backends.cudnn.deterministic = True
-    torch.cuda.manual_seed_all(1234)
 
     Trainer.__call__ = wrapper_logger(Trainer.__call__)  # type: ignore[assignment]
     # 第二步：对 Trainer.__call__ 函数装饰
diff --git a/docs/overrides/.icons/agent.svg b/docs/overrides/.icons/agent.svg
new file mode 100644
index 0000000..05f7521
--- /dev/null
+++ b/docs/overrides/.icons/agent.svg
@@ -0,0 +1,1 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 36 36"><path fill="#D4AB88" d="M13.483 21.579c-.58.32-2.256 1.279-3.282 2.204-.596.537-.518.928-.329 1.871l.017.085c.186.932 2.106 7.039 2.483 8.232l1.168-.291c.014-.621.043-2.801-.18-4.205-.074-.47-.149-1.036-.217-1.549-.07-.532-.138-1.035-.184-1.273-.077-.396.05-.65.411-.825.772-.374 1.437-.671 1.764-.814l-1.651-3.435zm10.673 5.408c-.533-.065-1.037-.128-1.278-.143-.403-.025-.618-.212-.695-.606-.168-.842-.288-1.56-.345-1.912l-3.74.733c.164.643.671 2.505 1.308 3.732.369.711.767.734 1.726.789l.088.005c.95.057 7.342-.266 8.593-.33l.013-1.205c-.597-.169-2.699-.746-4.114-.885-.476-.044-1.042-.114-1.556-.178z"/><path fill="#8899A6" d="M28.11 29.77l-.166-.306c-.021-.049.003-.106.054-.123.16-.055.483-.203.596-.262-.098.224-.21.582-.335.69-.048.041-.124.059-.149.001z"/><path fill="#BE1931" d="M30.53 27.396c-.13-.104-.427-.155-.613-.173-.129-.013-.464-.023-.746-.007-.127.007-.272.023-.331.04-.185.054-.266.131-.31.185-.053.065-.058.134-.046.169.049.143.284.39.386.625.034.078.054.155.047.226-.025.258-.354.633-.6.911-.148.167-.266.299-.271.35-.005.055.01.102.035.148v.002c.067.123.214.251.314.587.065.217.369 1.205.369 1.205.073.252.17.313.211.449.085.277.235 1.021.309 1.256.074.235.253.336.39.355.138.019.217-.013.217-.013s.322-.145.535-1.352c.154-.872.018-1.007-.098-2.317-.114-1.312.202-2.646.202-2.646z"/><path fill="#DD2E44" d="M30.261 29.344c-.219-.09-.72-.42-.86-.775-.127-.321-.213-1.188-.229-1.353.269-.024.682-.006.815.006.175.017.377.031.498.134l.046.04c-.001 0-.215 1.376-.27 1.948z"/><path fill="#CCD6DD" d="M28.337 30.337c-.075 0-.146-.042-.181-.115-.048-.1-.005-.219.095-.266l.319-.151c.104-.048.22-.004.267.095.047.1.005.219-.095.266l-.319.151c-.028.014-.058.02-.086.02zm.204.584c-.085 0-.163-.054-.19-.139-.034-.105.024-.218.13-.251l.354-.113c.1-.035.217.024.251.13.034.105-.024.218-.13.251l-.354.113c-.02.006-.04.009-.061.009zm.172.595c-.094 0-.177-.065-.196-.16-.021-.108.048-.214.156-.236l.332-.068c.114-.023.214.048.236.156.021.108-.048.214-.156.236l-.332.068c-.014.002-.027.004-.04.004zm2.59-3.873c-.006-.027-.025-.128-.095-.252-.023-.042-.077-.055-.131-.061-.067-.009-.56-.08-.569.02 0 0-.002.148-.072.25-.07.102-.118.352-.136.539-.018.186-.098.887-.08 1.379.018.492.17 1.383.178 1.68.007.297 0 .759-.063 1.101-.063.342-.131.646-.183.834-.067.242-.157.439-.26.578 0 0 .389-.026.687-.602.235-.454.367-.891.416-1.317.004-.032.105-1.168.154-1.736.035-.415.095-1.078.095-1.078.075-.786.093-1.185.059-1.335z"/><path fill="#8899A6" d="M11.438 32.516l.246-.246c.041-.034.102-.027.134.017.098.138.332.406.421.497-.243-.03-.618-.036-.757-.125-.054-.035-.093-.103-.044-.143z"/><path fill="#BE1931" d="M14.401 34.162c.063-.154.027-.453-.008-.637-.025-.127-.11-.452-.205-.717-.043-.119-.099-.254-.132-.306-.104-.162-.201-.217-.266-.244-.077-.033-.145-.018-.175.004-.123.088-.294.383-.489.548-.065.055-.133.095-.204.109-.254.049-.707-.159-1.044-.317-.202-.094-.362-.17-.412-.16-.054.011-.095.038-.132.075l-.002.001h.001c-.099.099-.18.277-.474.468-.189.123-1.051.696-1.051.696-.221.141-.252.252-.37.33-.242.16-.912.515-1.117.653-.204.138-.251.338-.23.475s.074.204.074.204.23.268 1.448.129c.88-.1.97-.269 2.194-.752s2.594-.559 2.594-.559z"/><path fill="#DD2E44" d="M12.457 34.457c.024-.236.198-.809.499-1.045.271-.212 1.079-.542 1.232-.604.1.251.2.652.225.783.033.173.078.37.013.516l-.025.056c0-.001-1.38.184-1.944.294z"/><path fill="#CCD6DD" d="M11.385 33.301c-.055 0-.109-.022-.149-.067L11 32.971c-.073-.082-.066-.209.016-.282.083-.073.207-.067.283.016l.235.263c.073.082.066.209-.016.282-.039.034-.085.051-.133.051zm-.529.406c-.063 0-.127-.031-.165-.088l-.209-.308c-.062-.092-.039-.216.053-.278.089-.061.216-.039.277.053l.209.308c.062.092.039.216-.053.278-.033.024-.072.035-.112.035zm-.57.325c-.071 0-.141-.038-.177-.106l-.159-.299c-.052-.098-.015-.219.083-.271.094-.052.218-.016.271.083l.159.299c.052.098.015.219-.083.271-.029.016-.062.023-.094.023zm4.098.941c.024-.013.116-.06.215-.162.033-.034.031-.089.022-.143-.011-.067-.083-.559-.181-.54 0 0-.143.04-.26.002-.117-.038-.371-.013-.555.022-.184.036-.878.158-1.345.315-.467.157-1.277.556-1.56.648-.283.091-.728.215-1.074.252-.346.037-.657.058-.851.062-.251.004-.466-.025-.628-.085 0 0 .136.366.772.488.502.097.959.099 1.381.025.032-.006 1.15-.231 1.708-.346.408-.084 1.06-.215 1.06-.215.774-.151 1.162-.247 1.296-.323z"/><path fill="#A0041E" d="M13.084 21.466c.738-.382 3.176-1.588 4.842-1.76.152-.016 1.526 3.904 1.341 3.963-.763.246-2.681.897-4.134 1.668-.154.082-.338.044-.433-.09-.431-.605-1.549-2.257-1.798-3.436-.029-.138.048-.276.182-.345z"/><path fill="#BE1931" d="M17.882 25.364c-.196-.795-.794-3.411-.586-5.049.019-.15 3.69-.685 3.69-.685.175-.021.6 3.084 1.005 4.655.043.166-.033.334-.181.395-.667.275-2.474.969-3.643.937-.136-.004-.249-.11-.285-.253z"/><path fill="#D4AB88" d="M14.801 11.245c-.114.64.259 1.431 1.738 1.741 1.479.31 2.554.273 3.324.218.653-.047.563.952.711 1.998.148 1.05.241 2.419.182 2.633-.1.361-.722.479-.439 1.123.283.643.329.995.942.79.613-.205 1.222-.427 1.396-.759.174-.332-.43-1.065-.439-1.463-.007-.294.293-3.972.275-4.405-.022-.528-.035-1.251-.884-1.582-.849-.331-4.108-1.174-4.972-1.342-.912-.177-1.683.199-1.834 1.048z"/><path fill="#D4AB88" d="M16.495 6.612c.31-.401.546-.872.661-1.455.409-2.083-.794-4.073-2.688-4.444-1.893-.372-4.483 1.029-4.425 3.151.057 2.107 1.229 5.214 2.993 4.69.466-.138.932-.29 1.375-.474.189.224.325 1.081.247 1.815-.2 1.87 2.381 2.065 2.579.192.157-1.482-.768-2.424-.742-3.475z"/><path fill="#963B22" d="M21.903 2.733c-1.311-.014-2.496-1.36-3.845-.501-.264.168-.456.337-.604.503-.234-.711-.636-1.367-1.299-1.798C15.507.515 14.712.2 13.961.285 9.384.801 9.632 4.111 9.98 4.421c.348.31 1.299-.572 2.002-.669.328-.046 1.041-.053 1.419.49.306.44.411 1.33.411 1.33s.012-1.012.041-1.368c.027-.332.356-.714.801-.222.387.428.19 1.268-.079 1.356-.207.068.129.575.648.961.946.704 1.681.428 1.786.389.106-.039.465-.118.648-1.333.026-.169.044-.35.055-.537.374.324.987.709 1.827.715.929.007 2.619-.466 3.586-1.621.744-.889 1.05-2.355 1.05-2.355-.395.738-1.283 1.187-2.272 1.176z"/><path fill="#DD2E44" d="M19.624 10.939c-.728-1.703-1.783-2.597-3.229-2.114-1.285.43-1.922 1.151-2.301 1.96-.184.393-.901 1.933-.466 2.933.851 1.956 2.243 5.863 2.488 7.059 0 0 .322.326 2.403.064 1.823-.23 2.732-.404 2.732-.404-.277-2.594-.452-6.747-1.627-9.498z"/><path fill="#D4AB88" d="M17.202 11.067c-.577-.372-1.499-.331-2.404.983-.905 1.314-1.308 2.376-1.568 3.148-.221.655-1.157.16-2.236-.122-1.083-.284-2.455-.752-2.639-.897-.311-.245-.172-.898-.914-.886-.742.013-1.103-.086-1.153.595-.05.681-.083 1.364.17 1.669.253.305 1.212.016 1.604.169.289.113 3.749 1.905 4.178 2.064.524.194 1.233.476 1.901-.216.668-.692 2.819-3.522 3.335-4.296.543-.814.491-1.718-.274-2.211z"/></svg>
diff --git a/docs/overrides/.icons/alg.svg b/docs/overrides/.icons/alg.svg
new file mode 100644
index 0000000..576dc51
--- /dev/null
+++ b/docs/overrides/.icons/alg.svg
@@ -0,0 +1,1 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 36 36"><ellipse fill="#F4900C" cx="33.5" cy="14.5" rx="2.5" ry="3.5"/><ellipse fill="#F4900C" cx="2.5" cy="14.5" rx="2.5" ry="3.5"/><path fill="#FFAC33" d="M34 19c0 .553-.447 1-1 1h-3c-.553 0-1-.447-1-1v-9c0-.552.447-1 1-1h3c.553 0 1 .448 1 1v9zM7 19c0 .553-.448 1-1 1H3c-.552 0-1-.447-1-1v-9c0-.552.448-1 1-1h3c.552 0 1 .448 1 1v9z"/><path fill="#FFCC4D" d="M28 5c0 2.761-4.478 4-10 4C12.477 9 8 7.761 8 5s4.477-5 10-5c5.522 0 10 2.239 10 5z"/><path fill="#F4900C" d="M25 4.083C25 5.694 21.865 7 18 7c-3.866 0-7-1.306-7-2.917 0-1.611 3.134-2.917 7-2.917 3.865 0 7 1.306 7 2.917z"/><path fill="#269" d="M30 5.5C30 6.881 28.881 7 27.5 7h-19C7.119 7 6 6.881 6 5.5S7.119 3 8.5 3h19C28.881 3 30 4.119 30 5.5z"/><path fill="#55ACEE" d="M30 6H6c-1.104 0-2 .896-2 2v26h28V8c0-1.104-.896-2-2-2z"/><path fill="#3B88C3" d="M35 33v-1c0-1.104-.896-2-2-2H22.071l-3.364 3.364c-.391.391-1.023.391-1.414 0L13.929 30H3c-1.104 0-2 .896-2 2v1c0 1.104-.104 2 1 2h32c1.104 0 1-.896 1-2z"/><circle fill="#FFF" cx="24.5" cy="14.5" r="4.5"/><circle fill="#DD2E44" cx="24.5" cy="14.5" r="2.721"/><circle fill="#FFF" cx="11.5" cy="14.5" r="4.5"/><path fill="#F5F8FA" d="M29 25.5c0 1.381-1.119 2.5-2.5 2.5h-17C8.119 28 7 26.881 7 25.5S8.119 23 9.5 23h17c1.381 0 2.5 1.119 2.5 2.5z"/><path fill="#CCD6DD" d="M17 23h2v5h-2zm-5 0h2v5h-2zm10 0h2v5h-2zM7 25.5c0 1.21.859 2.218 2 2.45v-4.9c-1.141.232-2 1.24-2 2.45zm20-2.45v4.899c1.141-.232 2-1.24 2-2.45s-.859-2.217-2-2.449z"/><circle fill="#DD2E44" cx="11.5" cy="14.5" r="2.721"/></svg>
diff --git a/docs/overrides/.icons/model.svg b/docs/overrides/.icons/model.svg
new file mode 100644
index 0000000..316d33e
--- /dev/null
+++ b/docs/overrides/.icons/model.svg
@@ -0,0 +1,1 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 36 36"><path fill="#EA596E" d="M29.896 26.667c.003.283-.07.653-.146.958-.531 2.145-2.889 4.552-6.208 4.333-3.008-.198-5.458-1.642-5.458-3.667s2.444-3.667 5.458-3.667 6.335.018 6.354 2.043z"/><path fill="#DD2E44" d="M23.542 24.964c-1.619 0-5.314.448-6.162.448-1.498 0-2.713.94-2.713 2.1 0 .558.286 1.062.744 1.438 0 0 1.006 1.009 2.818.525.793-.212 2.083-1.786 4.354-2.036 1.131-.125 3.25.75 6.974.771.16-.344.193-.583.193-.583 0-2.027-3.194-2.663-6.208-2.663z"/><path fill="#F4ABBA" d="M29.75 27.625s2.184-.443 3.542-2.229c1.583-2.083 1.375-4.312 1.375-4.312 1.604-3-.5-5.813-.5-5.813C33.958 12.104 32 10.792 32 10.792c-1.271-3.021-4.083-3.833-4.083-3.833-2.208-2.583-6.125-2.5-6.125-2.5s-3.67-1.345-8.708.167c-.833.25-3.625.833-5.667 2.083C.981 10.649.494 16.793.584 17.792 1.083 23.375 5 24.375 7.5 24.958c.583 1.583 2.729 4.5 6.583 3.417 4.75-.833 6.75-2.25 7.917-2.25s4.417 1.25 7.75 1.5z"/><g fill="#EA596E"><path d="M17.737 18.648c2.328-1.255 3.59-1.138 4.704-1.037.354.032.689.057 1.028.055 1.984-.045 3.591-.881 4.302-1.69.182-.208.161-.523-.046-.706-.206-.182-.521-.163-.706.045-.548.624-1.899 1.313-3.573 1.351-.3.009-.601-.021-.913-.05-1.195-.111-2.679-.247-5.271 1.152-.665.359-1.577.492-2.565.592-2.197-3.171-.875-5.933-.497-6.591.037.002.073.014.111.014.4 0 .802-.098 1.166-.304.24-.136.325-.441.189-.681s-.441-.325-.681-.189c-.64.362-1.482.153-1.88-.467-.149-.233-.458-.301-.69-.151-.232.149-.3.458-.151.69.237.371.571.65.948.837-.521 1.058-1.51 3.84.372 6.951-1.324.13-2.65.317-3.688.986-.67.432-1.355 1.064-1.878 1.791-.629-.108-2.932-.675-3.334-3.231.25-.194.452-.45.577-.766.102-.257-.024-.547-.281-.649s-.547.024-.649.281c-.089.225-.25.389-.454.461-.202.072-.431.048-.643-.07-.243-.135-.546-.046-.68.194-.134.241-.047.546.194.68.284.158.588.238.89.238.037 0 .072-.017.109-.019.476 2.413 2.383 3.473 3.732 3.794-.176.388-.295.788-.331 1.192-.024.274.179.518.454.542l.045.002c.256 0 .475-.195.498-.456.108-1.213 1.265-2.48 2.293-3.145.964-.621 2.375-.752 3.741-.879 1.325-.121 2.577-.237 3.558-.767zm12.866-1.504c-.131.242-.041.546.2.677.076.042.159.062.24.062.176 0 .347-.093.438-.259.019-.034 1.842-3.449-1.571-5.744-.228-.152-.539-.093-.694.136-.154.229-.093.54.136.694 2.644 1.778 1.309 4.326 1.251 4.434zM9.876 9.07c.156 0 .309-.072.406-.208 1.45-2.017 3.458-1.327 3.543-1.295.259.094.544-.038.64-.296.096-.259-.036-.546-.295-.642-.96-.356-3.177-.468-4.7 1.65-.161.223-.11.536.114.697.089.064.191.094.292.094zm13.072-1.888c2.225-.181 3.237 1.432 3.283 1.508.094.158.26.246.431.246.086 0 .174-.022.253-.069.238-.14.318-.446.179-.684-.054-.091-1.34-2.218-4.224-1.998-.275.021-.48.262-.459.538.022.275.271.484.537.459zm9.15 14.611c-.246-.014-.517.181-.539.457-.002.018-.161 1.719-1.91 2.294-.262.086-.405.368-.318.631.068.21.265.344.475.344.052 0 .104-.008.156-.025 2.372-.778 2.586-3.064 2.594-3.161.021-.276-.183-.518-.458-.54z"/><path d="M7.347 16.934c-.072.267.086.541.353.613.043.012.087.017.13.017.221 0 .422-.147.482-.37.19-.711.927-1.167 1.652-1.014.271.055.535-.116.592-.387.057-.27-.116-.535-.387-.592-.426-.09-.854-.048-1.248.086-1.166-1.994-.939-3.96-.936-3.981.036-.273-.156-.524-.429-.562-.272-.033-.524.154-.562.427-.013.097-.28 2.316 1.063 4.614-.334.301-.588.694-.71 1.149zm11.179-2.47c.407-.377 1.06-.37 1.455.015.098.094.223.141.349.141.131 0 .261-.051.358-.152.192-.198.188-.515-.01-.707-.23-.223-.505-.372-.797-.465.296-1.016.179-1.467-.096-2.312-.048-.147-.101-.311-.157-.498l-.03-.1c-.364-1.208-.605-2.005.087-3.13.145-.235.071-.543-.164-.688-.231-.144-.542-.073-.688.164-.928 1.508-.587 2.637-.192 3.944l.03.1c.059.194.113.364.163.517.247.761.322 1.016.02 1.936-.369.069-.722.236-1.01.504-.202.188-.213.505-.025.707.189.202.506.212.707.024zm6.365-2.985c.291 0 .585-.061.859-.191.25-.118.357-.416.239-.666-.117-.25-.419-.354-.665-.239-.502.236-1.121.031-1.384-.457-.131-.244-.436-.333-.677-.205-.243.13-.335.434-.204.677.18.336.448.601.76.785-.537 1.305-.232 2.691.017 3.426.07.209.265.34.474.34.053 0 .106-.008.16-.026.262-.088.402-.372.313-.633-.168-.498-.494-1.756-.002-2.826.038.002.073.015.11.015zm4.797 9.429c-.019-.275-.257-.492-.531-.467-1.014.072-1.878-.697-1.947-1.703-.02-.275-.265-.48-.533-.465-.275.019-.483.258-.465.533.041.59.266 1.122.608 1.555-.804.946-1.857 1.215-2.444 1.284-.519.062-.973.009-1.498-.053-.481-.055-1.025-.118-1.698-.098l-.005.001c-.02-.286-.088-.703-.305-1.05-.146-.233-.455-.304-.689-.158-.233.146-.305.455-.158.689.134.215.159.558.159.725-.504.181-.94.447-1.334.704-.704.458-1.259.82-2.094.632-.756-.173-1.513-.208-2.155-.118-.1-.251-.258-.551-.502-.782-.201-.19-.517-.182-.707.02-.189.2-.181.517.02.707.086.081.154.199.209.317-1.103.454-1.656 1.213-1.682 1.25-.16.224-.107.533.115.694.088.063.19.094.292.094.155 0 .308-.071.406-.205.005-.008.554-.743 1.637-1.04.56-.154 1.363-.141 2.146.037.219.05.422.067.619.07.093.218.129.477.134.573.016.266.236.472.499.472l.027-.001c.275-.015.486-.248.473-.523-.003-.05-.023-.346-.13-.686.461-.167.862-.428 1.239-.673.572-.373 1.113-.726 1.82-.749.592-.021 1.08.036 1.551.091.474.055.94.091 1.454.061.091.253.084.591.07.704-.033.273.16.523.434.559.021.003.042.004.063.004.247 0 .462-.183.495-.435.008-.062.059-.512-.059-.981.718-.193 1.608-.612 2.345-1.471.431.264.934.419 1.469.419.062 0 .125-.002.187-.006.275-.018.483-.255.465-.531z"/></g></svg>
diff --git a/docs/overrides/.icons/trainer.svg b/docs/overrides/.icons/trainer.svg
new file mode 100644
index 0000000..7b4f073
--- /dev/null
+++ b/docs/overrides/.icons/trainer.svg
@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Created with Vectornator (http://vectornator.io/) -->
+<svg height="100%" stroke-miterlimit="10" style="fill-rule:nonzero;clip-rule:evenodd;stroke-linecap:round;stroke-linejoin:round;" version="1.1" viewBox="0 0 1024 1024" width="100%" xml:space="preserve" xmlns="http://www.w3.org/2000/svg" xmlns:vectornator="http://vectornator.io" xmlns:xlink="http://www.w3.org/1999/xlink">
+<defs/>
+<g id="Layer-16" vectornator:layerName="Layer 16">
+<g opacity="1">
+<path d="M779.926 830.731L262.106 830.731C212.263 830.731 171.776 803.537 171.776 770.059L171.776 708.129C171.776 694.304 189.808 683.677 210.051 685.391L788.772 733.952C825.006 737.037 852.224 757.718 852.224 782.17C852.224 809.022 819.902 830.731 779.926 830.731Z" fill="#6a9fd3" fill-rule="nonzero" opacity="1" stroke="#1a2a3a" stroke-linecap="round" stroke-linejoin="round" stroke-width="46"/>
+<path d="M335.054 694.218L171.776 193.269L259.266 193.269" fill="none" opacity="1" stroke="#1a2a3a" stroke-linecap="round" stroke-linejoin="round" stroke-width="46"/>
+<path d="M198.989 674.134L271.836 528.233" fill="none" opacity="1" stroke="#1a2a3a" stroke-linecap="round" stroke-linejoin="round" stroke-width="46"/>
+<path d="M534.737 333.76L232.809 332.791" fill="none" opacity="1" stroke="#1a2a3a" stroke-linecap="round" stroke-linejoin="round" stroke-width="46"/>
+</g>
+</g>
+</svg>
diff --git a/docs/run.en.md b/docs/run.en.md
index b5987cf..2829b2c 100644
--- a/docs/run.en.md
+++ b/docs/run.en.md
@@ -3,34 +3,42 @@
 After dependencies are installed, you can run the algorithm file directly.
 
 ```shell
-python abcdrl/dqn.py \
+python abcdrl/dqn_torch.py \
     --env-id Cartpole-v1 \
-    --device "cuda:1" \ #(1)!
-    --total_timesteps 500000 \ #(2)!
+    --total_timesteps 500000 \ #(1)!
     --gamma 0.99 \
-    --learning-rate 2.5e-4 \ #(3)!
+    --learning-rate 2.5e-4 \ #(2)!
     --capture-video True \
-    --track \ #(4)!
+    --track \ #(3)!
     --wandb-project-name 'abcdrl' \
     --wandb-tags "['tag1', 'tag2']"
 ```
 
-1.  or `--device cuda:1`
-2.  The connector can use `_` or `-`
-3.  or `0.00025`
-4.  or `--track True`
+1.  The connector can use `_` or `-`
+2.  or `0.00025`
+3.  or `--track True`
+
+!!! example "Set specific GPU device"
+    - Using `gpu:0` and `gpu:1` 👇
+        - `CUDA_VISIBLE_DEVICES="0,1" python abcdrl/dqn_torch.py --cuda`
+    - Using `gpu:1` 👇
+        - `CUDA_VISIBLE_DEVICES="1" python abcdrl/dqn_torch.py --cuda`
+    - Using `cpu` only 👇
+        - `python abcdrl/dqn_torch.py --cuda False`
+        - `CUDA_VISIBLE_DEVICES="" python abcdrl/dqn_torch.py`
+        - `CUDA_VISIBLE_DEVICES="-1" python abcdrl/dqn_torch.py`
 
 Parameters in the algorithm file, consisting of two parts. The first part is the initialization parameters of `Trainer🔁`, and the second part is the parameters of the feature (`logger`, ...).
 
 === "Algorithm Parameters"
 
-    ```python title="abcdrl/dqn.py" linenums="206" hl_lines="4-11 13-16 18-19 21-23"
+    ```python title="abcdrl/dqn_torch.py" linenums="205" hl_lines="4-11 13-16 18-19 21-23"
     class Trainer:
         def __init__(
             self,
             exp_name: str | None = None,
             seed: int = 1,
-            device: str | torch.device = "auto",
+            cuda: bool = True,
             capture_video: bool = False,
             env_id: str = "CartPole-v1",
             num_envs: int = 1,
@@ -53,11 +61,12 @@ Parameters in the algorithm file, consisting of two parts. The first part is the
 
 === "Features Parameters"
 
-    ```python title="abcdrl/dqn.py" linenums="312" hl_lines="21-24"
+    ```python title="abcdrl/dqn_torch.py" linenums="310" hl_lines="22-25"
     def wrapper_logger(
         wrapped: Callable[..., Generator[dict[str, Any], None, None]]
     ) -> Callable[..., Generator[dict[str, Any], None, None]]:
         import wandb
+        from torch.utils.tensorboard import SummaryWriter
 
         def setup_video_monitor() -> None:
             vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -83,4 +92,4 @@ Parameters in the algorithm file, consisting of two parts. The first part is the
     ```
 
 !!! note
-    You can use the `python abcdrl/dqn.py --help` command to view algorithm parameters and the `python abcdrl/dqn.py __call__ --help` command to view features parameters.
+    You can use the `python abcdrl/dqn_torch.py --help` command to view algorithm parameters and the `python abcdrl/dqn_torch.py __call__ --help` command to view features parameters.
diff --git a/docs/run.zh.md b/docs/run.zh.md
index 32675ff..3001ae8 100644
--- a/docs/run.zh.md
+++ b/docs/run.zh.md
@@ -3,34 +3,42 @@
 安装好所有依赖后，即可直接运行算法文件。
 
 ```shell
-python abcdrl/dqn.py \
+python abcdrl/dqn_torch.py \
     --env-id Cartpole-v1 \
-    --device "cuda:1" \ #(1)!
-    --total_timesteps 500000 \ #(2)!
+    --total_timesteps 500000 \ #(1)!
     --gamma 0.99 \
-    --learning-rate 2.5e-4 \ #(3)!
+    --learning-rate 2.5e-4 \ #(2)!
     --capture-video True \
-    --track \ #(4)!
+    --track \ #(3)!
     --wandb-project-name 'abcdrl' \
     --wandb-tags "['tag1', 'tag2']"
 ```
 
-1.  或 `--device cuda:1`
-2.  连接符可以使用 `_` 或 `-`
-3.  或 `0.00025`
-4.  或 `--track True`
+1.  连接符可以使用 `_` 或 `-`
+2.  或 `0.00025`
+3.  或 `--track True`
+
+!!! example "指定 GPU 设备"
+    - 使用 `gpu:0` 和 `gpu:1` 👇
+        - `CUDA_VISIBLE_DEVICES="0,1" python abcdrl/dqn_torch.py --cuda`
+    - 使用 `gpu:1` 👇
+        - `CUDA_VISIBLE_DEVICES="1" python abcdrl/dqn_torch.py --cuda`
+    - 仅使用 `cpu` 👇
+        - `python abcdrl/dqn_torch.py --cuda False`
+        - `CUDA_VISIBLE_DEVICES="" python abcdrl/dqn_torch.py`
+        - `CUDA_VISIBLE_DEVICES="-1" python abcdrl/dqn_torch.py`
 
 算法文件中的参数，由两部分组成。第一部分是算法主体 `Trainer🔁` 的参数，第二部分是功能（`logger`, ...）的参数。
 
 === "算法参数"
 
-    ```python title="abcdrl/dqn.py" linenums="206" hl_lines="4-11 13-16 18-19 21-23"
+    ```python title="abcdrl/dqn_torch.py" linenums="205" hl_lines="4-11 13-16 18-19 21-23"
     class Trainer:
         def __init__(
             self,
             exp_name: str | None = None,
             seed: int = 1,
-            device: str | torch.device = "auto",
+            cuda: bool = True,
             capture_video: bool = False,
             env_id: str = "CartPole-v1",
             num_envs: int = 1,
@@ -53,11 +61,12 @@ python abcdrl/dqn.py \
 
 === "功能参数"
 
-    ```python title="abcdrl/dqn.py" linenums="312" hl_lines="21-24"
+    ```python title="abcdrl/dqn_torch.py" linenums="310" hl_lines="22-25"
     def wrapper_logger(
         wrapped: Callable[..., Generator[dict[str, Any], None, None]]
     ) -> Callable[..., Generator[dict[str, Any], None, None]]:
         import wandb
+        from torch.utils.tensorboard import SummaryWriter
 
         def setup_video_monitor() -> None:
             vcr = gym.wrappers.monitoring.video_recorder.VideoRecorder
@@ -83,4 +92,4 @@ python abcdrl/dqn.py \
     ```
 
 !!! note
-    可使用 `python abcdrl/dqn.py --help` 命令查看算法参数，使用 `python abcdrl/dqn.py __call__ --help` 命令查看功能参数。
+    可使用 `python abcdrl/dqn_torch.py --help` 命令查看算法参数，使用 `python abcdrl/dqn_torch.py __call__ --help` 命令查看功能参数。
diff --git a/example_eval_model.py b/example_eval_model.py
index 234e737..3dc3f7a 100644
--- a/example_eval_model.py
+++ b/example_eval_model.py
@@ -17,14 +17,13 @@ def __init__(
         model_path: str,
         env_id: str = "CartPole-v1",
         total_timesteps: int = 1_000_0,
-        device: str = "auto",
+        cuda: bool = True,
         seed: int = 1,
         capture_video: bool = False,
     ) -> None:
         self.kwargs = locals()
         self.kwargs.pop("self")
-        if self.kwargs["device"] == "auto":
-            self.kwargs["device"] = "cuda" if torch.cuda.is_available() else "cpu"
+        self.kwargs["device"] = "cuda" if self.kwargs["cuda"] and torch.cuda.is_available() else "cpu"
 
         self.eval_env = gym.vector.SyncVectorEnv([self._make_env(0)])  # type: ignore[arg-type]
         self.eval_obs, _ = self.eval_env.reset(seed=0)
diff --git a/example_tuner_sweep.py b/example_tuner_sweep.py
index 64811f6..4f6aa1b 100644
--- a/example_tuner_sweep.py
+++ b/example_tuner_sweep.py
@@ -5,7 +5,7 @@
 import fire
 import wandb
 
-import abcdrl
+from abcdrl import dqn_torch
 
 sweep_configuration = {
     "method": "random",
@@ -24,9 +24,9 @@
 
 def tune_agent() -> None:
     with wandb.init(name=f"{sweep_configuration['name']}__{int(time.time())}") as writer:  # type: ignore[union-attr]
-        wandb.save("abcdrl/dqn.py")
+        wandb.save("abcdrl/dqn_torch.py")
 
-        trainer = abcdrl.dqn.Trainer(
+        trainer = dqn_torch.Trainer(
             env_id=wandb.config.env_id,
             learning_rate=wandb.config.learning_rate,
             batch_size=wandb.config.batch_size,
diff --git a/poetry.lock b/poetry.lock
index 5dd65b2..def76a9 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -60,6 +60,27 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "astunparse"
+version = "1.6.3"
+description = "An AST unparser for Python"
+category = "main"
+optional = true
+python-versions = "*"
+files = [
+    {file = "astunparse-1.6.3-py2.py3-none-any.whl", hash = "sha256:c2652417f2c8b5bb325c885ae329bdf3f86424075c4fd1a128674bc6fba4b8e8"},
+    {file = "astunparse-1.6.3.tar.gz", hash = "sha256:5ad93a8456f0d084c3456d059fd9a92cce667963232cbf763eac3bc5b7940872"},
+]
+
+[package.dependencies]
+six = ">=1.6.1,<2.0"
+wheel = ">=0.23.0,<1.0"
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "attrs"
 version = "22.2.0"
@@ -86,20 +107,21 @@ reference = "tsinghua"
 
 [[package]]
 name = "autorom"
-version = "0.4.2"
+version = "0.5.0"
 description = "Automated installation of Atari ROMs for Gym/ALE-Py"
 category = "main"
 optional = true
 python-versions = ">=3.6"
 files = [
-    {file = "AutoROM-0.4.2-py3-none-any.whl", hash = "sha256:719c9d363ef08391fdb7003d70df235b68f36de628d289a946c4a59a3adefa13"},
-    {file = "AutoROM-0.4.2.tar.gz", hash = "sha256:b426a39bc0ee3781c7791f28963a9b2e4385b6421eeaf2f368edc00c761d428a"},
+    {file = "AutoROM-0.5.0-py3-none-any.whl", hash = "sha256:9cfe4630de75fad33fd184541e1155e3d9025be6ac6d1ce215c5f67afc8ca26e"},
+    {file = "AutoROM-0.5.0.tar.gz", hash = "sha256:f639d1d8fd57bd14af11b0aa6328087f295d7b9fd731f238ca3aaf160c9d8cfe"},
 ]
 
 [package.dependencies]
 "AutoROM.accept-rom-license" = {version = "*", optional = true, markers = "extra == \"accept-rom-license\""}
 click = "*"
 importlib-resources = {version = "*", markers = "python_version < \"3.9\""}
+libtorrent = "*"
 requests = "*"
 tqdm = "*"
 
@@ -616,6 +638,23 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "flatbuffers"
+version = "22.12.6"
+description = "The FlatBuffers serialization format for Python"
+category = "main"
+optional = true
+python-versions = "*"
+files = [
+    {file = "flatbuffers-22.12.6-py2.py3-none-any.whl", hash = "sha256:b12a3214e73f325ecd392503e821ac43a791181356f020cc3657ba829b4947c8"},
+    {file = "flatbuffers-22.12.6.tar.gz", hash = "sha256:27f67c6fb102d41c911c26867bda71f1f8622176ac072fa30f668f4d023b5826"},
+]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "free-mujoco-py"
 version = "2.1.6"
@@ -641,6 +680,23 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "gast"
+version = "0.4.0"
+description = "Python AST that abstracts the underlying Python version"
+category = "main"
+optional = true
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+files = [
+    {file = "gast-0.4.0-py3-none-any.whl", hash = "sha256:b7adcdd5adbebf1adf17378da5ba3f543684dbec47b1cda1f3997e573cd542c4"},
+    {file = "gast-0.4.0.tar.gz", hash = "sha256:40feb7b8b8434785585ab224d1568b857edb18297e5a3047f1ba012bc83b42c1"},
+]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "ghp-import"
 version = "2.1.0"
@@ -779,6 +835,27 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "google-pasta"
+version = "0.2.0"
+description = "pasta is an AST-based Python refactoring library"
+category = "main"
+optional = true
+python-versions = "*"
+files = [
+    {file = "google-pasta-0.2.0.tar.gz", hash = "sha256:c9f2c8dfc8f96d0d5808299920721be30c9eec37f2389f28904f454565c8a16e"},
+    {file = "google_pasta-0.2.0-py2-none-any.whl", hash = "sha256:4612951da876b1a10fe3960d7226f0c7682cf901e16ac06e473b267a5afa8954"},
+    {file = "google_pasta-0.2.0-py3-none-any.whl", hash = "sha256:b32482794a366b5366a32c92a9a9201b107821889935a02b3e51f6b432ea84ed"},
+]
+
+[package.dependencies]
+six = "*"
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "grpcio"
 version = "1.51.1"
@@ -898,6 +975,44 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "h5py"
+version = "3.7.0"
+description = "Read and write HDF5 files from Python"
+category = "main"
+optional = true
+python-versions = ">=3.7"
+files = [
+    {file = "h5py-3.7.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:d77af42cb751ad6cc44f11bae73075a07429a5cf2094dfde2b1e716e059b3911"},
+    {file = "h5py-3.7.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:63beb8b7b47d0896c50de6efb9a1eaa81dbe211f3767e7dd7db159cea51ba37a"},
+    {file = "h5py-3.7.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:04e2e1e2fc51b8873e972a08d2f89625ef999b1f2d276199011af57bb9fc7851"},
+    {file = "h5py-3.7.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f73307c876af49aa869ec5df1818e9bb0bdcfcf8a5ba773cc45a4fba5a286a5c"},
+    {file = "h5py-3.7.0-cp310-cp310-win_amd64.whl", hash = "sha256:f514b24cacdd983e61f8d371edac8c1b780c279d0acb8485639e97339c866073"},
+    {file = "h5py-3.7.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:43fed4d13743cf02798a9a03a360a88e589d81285e72b83f47d37bb64ed44881"},
+    {file = "h5py-3.7.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:c038399ce09a58ff8d89ec3e62f00aa7cb82d14f34e24735b920e2a811a3a426"},
+    {file = "h5py-3.7.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:03d64fb86bb86b978928bad923b64419a23e836499ec6363e305ad28afd9d287"},
+    {file = "h5py-3.7.0-cp37-cp37m-win_amd64.whl", hash = "sha256:e5b7820b75f9519499d76cc708e27242ccfdd9dfb511d6deb98701961d0445aa"},
+    {file = "h5py-3.7.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:a9351d729ea754db36d175098361b920573fdad334125f86ac1dd3a083355e20"},
+    {file = "h5py-3.7.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:6776d896fb90c5938de8acb925e057e2f9f28755f67ec3edcbc8344832616c38"},
+    {file = "h5py-3.7.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:0a047fddbe6951bce40e9cde63373c838a978c5e05a011a682db9ba6334b8e85"},
+    {file = "h5py-3.7.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0798a9c0ff45f17d0192e4d7114d734cac9f8b2b2c76dd1d923c4d0923f27bb6"},
+    {file = "h5py-3.7.0-cp38-cp38-win_amd64.whl", hash = "sha256:0d8de8cb619fc597da7cf8cdcbf3b7ff8c5f6db836568afc7dc16d21f59b2b49"},
+    {file = "h5py-3.7.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:f084bbe816907dfe59006756f8f2d16d352faff2d107f4ffeb1d8de126fc5dc7"},
+    {file = "h5py-3.7.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:1fcb11a2dc8eb7ddcae08afd8fae02ba10467753a857fa07a404d700a93f3d53"},
+    {file = "h5py-3.7.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:ed43e2cc4f511756fd664fb45d6b66c3cbed4e3bd0f70e29c37809b2ae013c44"},
+    {file = "h5py-3.7.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9e7535df5ee3dc3e5d1f408fdfc0b33b46bc9b34db82743c82cd674d8239b9ad"},
+    {file = "h5py-3.7.0-cp39-cp39-win_amd64.whl", hash = "sha256:9e2ad2aa000f5b1e73b5dfe22f358ca46bf1a2b6ca394d9659874d7fc251731a"},
+    {file = "h5py-3.7.0.tar.gz", hash = "sha256:3fcf37884383c5da64846ab510190720027dca0768def34dd8dcb659dbe5cbf3"},
+]
+
+[package.dependencies]
+numpy = ">=1.14.5"
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "identify"
 version = "2.5.11"
@@ -1107,6 +1222,47 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "keras"
+version = "2.11.0"
+description = "Deep learning for humans."
+category = "main"
+optional = true
+python-versions = ">=3.7"
+files = [
+    {file = "keras-2.11.0-py2.py3-none-any.whl", hash = "sha256:38c6fff0ea9a8b06a2717736565c92a73c8cd9b1c239e7125ccb188b7848f65e"},
+]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
+[[package]]
+name = "libclang"
+version = "14.0.6"
+description = "Clang Python Bindings, mirrored from the official LLVM repo: https://github.com/llvm/llvm-project/tree/main/clang/bindings/python, to make the installation process easier."
+category = "main"
+optional = true
+python-versions = "*"
+files = [
+    {file = "libclang-14.0.6-py2.py3-none-macosx_10_9_x86_64.whl", hash = "sha256:8791cf3c3b087c373a6d61e9199da7a541da922c9ddcfed1122090586b996d6e"},
+    {file = "libclang-14.0.6-py2.py3-none-macosx_11_0_arm64.whl", hash = "sha256:7b06fc76bd1e67c8b04b5719bf2ac5d6a323b289b245dfa9e468561d99538188"},
+    {file = "libclang-14.0.6-py2.py3-none-manylinux1_x86_64.whl", hash = "sha256:e429853939423f276a25140b0b702442d7da9a09e001c05e48df888336947614"},
+    {file = "libclang-14.0.6-py2.py3-none-manylinux2010_x86_64.whl", hash = "sha256:206d2789e4450a37d054e63b70451a6fc1873466397443fa13de2b3d4adb2796"},
+    {file = "libclang-14.0.6-py2.py3-none-manylinux2014_aarch64.whl", hash = "sha256:e2add1703129b2abe066fb1890afa880870a89fd6ab4ec5d2a7a8dc8d271677e"},
+    {file = "libclang-14.0.6-py2.py3-none-manylinux2014_armv7l.whl", hash = "sha256:5dd3c6fca1b007d308a4114afa8e4e9d32f32b2572520701d45fcc626ac5cd6c"},
+    {file = "libclang-14.0.6-py2.py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:cfb0e892ebb5dff6bd498ab5778adb8581f26a00fd8347b3c76c989fe2fd04f7"},
+    {file = "libclang-14.0.6-py2.py3-none-win_amd64.whl", hash = "sha256:ea03c12675151837660cdd5dce65bd89320896ac3421efef43a36678f113ce95"},
+    {file = "libclang-14.0.6-py2.py3-none-win_arm64.whl", hash = "sha256:2e4303e04517fcd11173cb2e51a7070eed71e16ef45d4e26a82c5e881cac3d27"},
+    {file = "libclang-14.0.6.tar.gz", hash = "sha256:9052a8284d8846984f6fa826b1d7460a66d3b23a486d782633b42b6e3b418789"},
+]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "libtorrent"
 version = "2.0.7"
@@ -1701,11 +1857,35 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "opt-einsum"
+version = "3.3.0"
+description = "Optimizing numpys einsum function"
+category = "main"
+optional = true
+python-versions = ">=3.5"
+files = [
+    {file = "opt_einsum-3.3.0-py3-none-any.whl", hash = "sha256:2455e59e3947d3c275477df7f5205b30635e266fe6dc300e3d9f9646bfcea147"},
+    {file = "opt_einsum-3.3.0.tar.gz", hash = "sha256:59f6475f77bbc37dcf7cd748519c0ec60722e91e63ca114e68821c0c54a46549"},
+]
+
+[package.dependencies]
+numpy = ">=1.7"
+
+[package.extras]
+docs = ["numpydoc", "sphinx (==1.2.3)", "sphinx-rtd-theme", "sphinxcontrib-napoleon"]
+tests = ["pytest", "pytest-cov", "pytest-pep8"]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "packaging"
 version = "22.0"
 description = "Core utilities for Python packages"
-category = "dev"
+category = "main"
 optional = false
 python-versions = ">=3.7"
 files = [
@@ -1736,77 +1916,79 @@ reference = "tsinghua"
 
 [[package]]
 name = "pillow"
-version = "9.3.0"
+version = "9.4.0"
 description = "Python Imaging Library (Fork)"
 category = "main"
 optional = false
 python-versions = ">=3.7"
 files = [
-    {file = "Pillow-9.3.0-1-cp37-cp37m-win32.whl", hash = "sha256:e6ea6b856a74d560d9326c0f5895ef8050126acfdc7ca08ad703eb0081e82b74"},
-    {file = "Pillow-9.3.0-1-cp37-cp37m-win_amd64.whl", hash = "sha256:32a44128c4bdca7f31de5be641187367fe2a450ad83b833ef78910397db491aa"},
-    {file = "Pillow-9.3.0-cp310-cp310-macosx_10_10_x86_64.whl", hash = "sha256:0b7257127d646ff8676ec8a15520013a698d1fdc48bc2a79ba4e53df792526f2"},
-    {file = "Pillow-9.3.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:b90f7616ea170e92820775ed47e136208e04c967271c9ef615b6fbd08d9af0e3"},
-    {file = "Pillow-9.3.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:68943d632f1f9e3dce98908e873b3a090f6cba1cbb1b892a9e8d97c938871fbe"},
-    {file = "Pillow-9.3.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:be55f8457cd1eac957af0c3f5ece7bc3f033f89b114ef30f710882717670b2a8"},
-    {file = "Pillow-9.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5d77adcd56a42d00cc1be30843d3426aa4e660cab4a61021dc84467123f7a00c"},
-    {file = "Pillow-9.3.0-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:829f97c8e258593b9daa80638aee3789b7df9da5cf1336035016d76f03b8860c"},
-    {file = "Pillow-9.3.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:801ec82e4188e935c7f5e22e006d01611d6b41661bba9fe45b60e7ac1a8f84de"},
-    {file = "Pillow-9.3.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:871b72c3643e516db4ecf20efe735deb27fe30ca17800e661d769faab45a18d7"},
-    {file = "Pillow-9.3.0-cp310-cp310-win32.whl", hash = "sha256:655a83b0058ba47c7c52e4e2df5ecf484c1b0b0349805896dd350cbc416bdd91"},
-    {file = "Pillow-9.3.0-cp310-cp310-win_amd64.whl", hash = "sha256:9f47eabcd2ded7698106b05c2c338672d16a6f2a485e74481f524e2a23c2794b"},
-    {file = "Pillow-9.3.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:57751894f6618fd4308ed8e0c36c333e2f5469744c34729a27532b3db106ee20"},
-    {file = "Pillow-9.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7db8b751ad307d7cf238f02101e8e36a128a6cb199326e867d1398067381bff4"},
-    {file = "Pillow-9.3.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3033fbe1feb1b59394615a1cafaee85e49d01b51d54de0cbf6aa8e64182518a1"},
-    {file = "Pillow-9.3.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:22b012ea2d065fd163ca096f4e37e47cd8b59cf4b0fd47bfca6abb93df70b34c"},
-    {file = "Pillow-9.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b9a65733d103311331875c1dca05cb4606997fd33d6acfed695b1232ba1df193"},
-    {file = "Pillow-9.3.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:502526a2cbfa431d9fc2a079bdd9061a2397b842bb6bc4239bb176da00993812"},
-    {file = "Pillow-9.3.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:90fb88843d3902fe7c9586d439d1e8c05258f41da473952aa8b328d8b907498c"},
-    {file = "Pillow-9.3.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:89dca0ce00a2b49024df6325925555d406b14aa3efc2f752dbb5940c52c56b11"},
-    {file = "Pillow-9.3.0-cp311-cp311-win32.whl", hash = "sha256:3168434d303babf495d4ba58fc22d6604f6e2afb97adc6a423e917dab828939c"},
-    {file = "Pillow-9.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:18498994b29e1cf86d505edcb7edbe814d133d2232d256db8c7a8ceb34d18cef"},
-    {file = "Pillow-9.3.0-cp37-cp37m-macosx_10_10_x86_64.whl", hash = "sha256:772a91fc0e03eaf922c63badeca75e91baa80fe2f5f87bdaed4280662aad25c9"},
-    {file = "Pillow-9.3.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:afa4107d1b306cdf8953edde0534562607fe8811b6c4d9a486298ad31de733b2"},
-    {file = "Pillow-9.3.0-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b4012d06c846dc2b80651b120e2cdd787b013deb39c09f407727ba90015c684f"},
-    {file = "Pillow-9.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:77ec3e7be99629898c9a6d24a09de089fa5356ee408cdffffe62d67bb75fdd72"},
-    {file = "Pillow-9.3.0-cp37-cp37m-manylinux_2_28_aarch64.whl", hash = "sha256:6c738585d7a9961d8c2821a1eb3dcb978d14e238be3d70f0a706f7fa9316946b"},
-    {file = "Pillow-9.3.0-cp37-cp37m-manylinux_2_28_x86_64.whl", hash = "sha256:828989c45c245518065a110434246c44a56a8b2b2f6347d1409c787e6e4651ee"},
-    {file = "Pillow-9.3.0-cp37-cp37m-win32.whl", hash = "sha256:82409ffe29d70fd733ff3c1025a602abb3e67405d41b9403b00b01debc4c9a29"},
-    {file = "Pillow-9.3.0-cp37-cp37m-win_amd64.whl", hash = "sha256:41e0051336807468be450d52b8edd12ac60bebaa97fe10c8b660f116e50b30e4"},
-    {file = "Pillow-9.3.0-cp38-cp38-macosx_10_10_x86_64.whl", hash = "sha256:b03ae6f1a1878233ac620c98f3459f79fd77c7e3c2b20d460284e1fb370557d4"},
-    {file = "Pillow-9.3.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:4390e9ce199fc1951fcfa65795f239a8a4944117b5935a9317fb320e7767b40f"},
-    {file = "Pillow-9.3.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:40e1ce476a7804b0fb74bcfa80b0a2206ea6a882938eaba917f7a0f004b42502"},
-    {file = "Pillow-9.3.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a0a06a052c5f37b4ed81c613a455a81f9a3a69429b4fd7bb913c3fa98abefc20"},
-    {file = "Pillow-9.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:03150abd92771742d4a8cd6f2fa6246d847dcd2e332a18d0c15cc75bf6703040"},
-    {file = "Pillow-9.3.0-cp38-cp38-manylinux_2_28_aarch64.whl", hash = "sha256:15c42fb9dea42465dfd902fb0ecf584b8848ceb28b41ee2b58f866411be33f07"},
-    {file = "Pillow-9.3.0-cp38-cp38-manylinux_2_28_x86_64.whl", hash = "sha256:51e0e543a33ed92db9f5ef69a0356e0b1a7a6b6a71b80df99f1d181ae5875636"},
-    {file = "Pillow-9.3.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:3dd6caf940756101205dffc5367babf288a30043d35f80936f9bfb37f8355b32"},
-    {file = "Pillow-9.3.0-cp38-cp38-win32.whl", hash = "sha256:f1ff2ee69f10f13a9596480335f406dd1f70c3650349e2be67ca3139280cade0"},
-    {file = "Pillow-9.3.0-cp38-cp38-win_amd64.whl", hash = "sha256:276a5ca930c913f714e372b2591a22c4bd3b81a418c0f6635ba832daec1cbcfc"},
-    {file = "Pillow-9.3.0-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:73bd195e43f3fadecfc50c682f5055ec32ee2c933243cafbfdec69ab1aa87cad"},
-    {file = "Pillow-9.3.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:1c7c8ae3864846fc95f4611c78129301e203aaa2af813b703c55d10cc1628535"},
-    {file = "Pillow-9.3.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2e0918e03aa0c72ea56edbb00d4d664294815aa11291a11504a377ea018330d3"},
-    {file = "Pillow-9.3.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b0915e734b33a474d76c28e07292f196cdf2a590a0d25bcc06e64e545f2d146c"},
-    {file = "Pillow-9.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:af0372acb5d3598f36ec0914deed2a63f6bcdb7b606da04dc19a88d31bf0c05b"},
-    {file = "Pillow-9.3.0-cp39-cp39-manylinux_2_28_aarch64.whl", hash = "sha256:ad58d27a5b0262c0c19b47d54c5802db9b34d38bbf886665b626aff83c74bacd"},
-    {file = "Pillow-9.3.0-cp39-cp39-manylinux_2_28_x86_64.whl", hash = "sha256:97aabc5c50312afa5e0a2b07c17d4ac5e865b250986f8afe2b02d772567a380c"},
-    {file = "Pillow-9.3.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:9aaa107275d8527e9d6e7670b64aabaaa36e5b6bd71a1015ddd21da0d4e06448"},
-    {file = "Pillow-9.3.0-cp39-cp39-win32.whl", hash = "sha256:bac18ab8d2d1e6b4ce25e3424f709aceef668347db8637c2296bcf41acb7cf48"},
-    {file = "Pillow-9.3.0-cp39-cp39-win_amd64.whl", hash = "sha256:b472b5ea442148d1c3e2209f20f1e0bb0eb556538690fa70b5e1f79fa0ba8dc2"},
-    {file = "Pillow-9.3.0-pp37-pypy37_pp73-macosx_10_10_x86_64.whl", hash = "sha256:ab388aaa3f6ce52ac1cb8e122c4bd46657c15905904b3120a6248b5b8b0bc228"},
-    {file = "Pillow-9.3.0-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:dbb8e7f2abee51cef77673be97760abff1674ed32847ce04b4af90f610144c7b"},
-    {file = "Pillow-9.3.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bca31dd6014cb8b0b2db1e46081b0ca7d936f856da3b39744aef499db5d84d02"},
-    {file = "Pillow-9.3.0-pp37-pypy37_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:c7025dce65566eb6e89f56c9509d4f628fddcedb131d9465cacd3d8bac337e7e"},
-    {file = "Pillow-9.3.0-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:ebf2029c1f464c59b8bdbe5143c79fa2045a581ac53679733d3a91d400ff9efb"},
-    {file = "Pillow-9.3.0-pp38-pypy38_pp73-macosx_10_10_x86_64.whl", hash = "sha256:b59430236b8e58840a0dfb4099a0e8717ffb779c952426a69ae435ca1f57210c"},
-    {file = "Pillow-9.3.0-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:12ce4932caf2ddf3e41d17fc9c02d67126935a44b86df6a206cf0d7161548627"},
-    {file = "Pillow-9.3.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ae5331c23ce118c53b172fa64a4c037eb83c9165aba3a7ba9ddd3ec9fa64a699"},
-    {file = "Pillow-9.3.0-pp38-pypy38_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:0b07fffc13f474264c336298d1b4ce01d9c5a011415b79d4ee5527bb69ae6f65"},
-    {file = "Pillow-9.3.0-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:073adb2ae23431d3b9bcbcff3fe698b62ed47211d0716b067385538a1b0f28b8"},
-    {file = "Pillow-9.3.0.tar.gz", hash = "sha256:c935a22a557a560108d780f9a0fc426dd7459940dc54faa49d83249c8d3e760f"},
+    {file = "Pillow-9.4.0-cp310-cp310-macosx_10_10_x86_64.whl", hash = "sha256:2968c58feca624bb6c8502f9564dd187d0e1389964898f5e9e1fbc8533169157"},
+    {file = "Pillow-9.4.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:c5c1362c14aee73f50143d74389b2c158707b4abce2cb055b7ad37ce60738d47"},
+    {file = "Pillow-9.4.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bd752c5ff1b4a870b7661234694f24b1d2b9076b8bf337321a814c612665f343"},
+    {file = "Pillow-9.4.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:9a3049a10261d7f2b6514d35bbb7a4dfc3ece4c4de14ef5876c4b7a23a0e566d"},
+    {file = "Pillow-9.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:16a8df99701f9095bea8a6c4b3197da105df6f74e6176c5b410bc2df2fd29a57"},
+    {file = "Pillow-9.4.0-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:94cdff45173b1919350601f82d61365e792895e3c3a3443cf99819e6fbf717a5"},
+    {file = "Pillow-9.4.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:ed3e4b4e1e6de75fdc16d3259098de7c6571b1a6cc863b1a49e7d3d53e036070"},
+    {file = "Pillow-9.4.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:d5b2f8a31bd43e0f18172d8ac82347c8f37ef3e0b414431157718aa234991b28"},
+    {file = "Pillow-9.4.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:09b89ddc95c248ee788328528e6a2996e09eaccddeeb82a5356e92645733be35"},
+    {file = "Pillow-9.4.0-cp310-cp310-win32.whl", hash = "sha256:f09598b416ba39a8f489c124447b007fe865f786a89dbfa48bb5cf395693132a"},
+    {file = "Pillow-9.4.0-cp310-cp310-win_amd64.whl", hash = "sha256:f6e78171be3fb7941f9910ea15b4b14ec27725865a73c15277bc39f5ca4f8391"},
+    {file = "Pillow-9.4.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:3fa1284762aacca6dc97474ee9c16f83990b8eeb6697f2ba17140d54b453e133"},
+    {file = "Pillow-9.4.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:eaef5d2de3c7e9b21f1e762f289d17b726c2239a42b11e25446abf82b26ac132"},
+    {file = "Pillow-9.4.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a4dfdae195335abb4e89cc9762b2edc524f3c6e80d647a9a81bf81e17e3fb6f0"},
+    {file = "Pillow-9.4.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6abfb51a82e919e3933eb137e17c4ae9c0475a25508ea88993bb59faf82f3b35"},
+    {file = "Pillow-9.4.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:451f10ef963918e65b8869e17d67db5e2f4ab40e716ee6ce7129b0cde2876eab"},
+    {file = "Pillow-9.4.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:6663977496d616b618b6cfa43ec86e479ee62b942e1da76a2c3daa1c75933ef4"},
+    {file = "Pillow-9.4.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:60e7da3a3ad1812c128750fc1bc14a7ceeb8d29f77e0a2356a8fb2aa8925287d"},
+    {file = "Pillow-9.4.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:19005a8e58b7c1796bc0167862b1f54a64d3b44ee5d48152b06bb861458bc0f8"},
+    {file = "Pillow-9.4.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:f715c32e774a60a337b2bb8ad9839b4abf75b267a0f18806f6f4f5f1688c4b5a"},
+    {file = "Pillow-9.4.0-cp311-cp311-win32.whl", hash = "sha256:b222090c455d6d1a64e6b7bb5f4035c4dff479e22455c9eaa1bdd4c75b52c80c"},
+    {file = "Pillow-9.4.0-cp311-cp311-win_amd64.whl", hash = "sha256:ba6612b6548220ff5e9df85261bddc811a057b0b465a1226b39bfb8550616aee"},
+    {file = "Pillow-9.4.0-cp37-cp37m-macosx_10_10_x86_64.whl", hash = "sha256:5f532a2ad4d174eb73494e7397988e22bf427f91acc8e6ebf5bb10597b49c493"},
+    {file = "Pillow-9.4.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5dd5a9c3091a0f414a963d427f920368e2b6a4c2f7527fdd82cde8ef0bc7a327"},
+    {file = "Pillow-9.4.0-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ef21af928e807f10bf4141cad4746eee692a0dd3ff56cfb25fce076ec3cc8abe"},
+    {file = "Pillow-9.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:847b114580c5cc9ebaf216dd8c8dbc6b00a3b7ab0131e173d7120e6deade1f57"},
+    {file = "Pillow-9.4.0-cp37-cp37m-manylinux_2_28_aarch64.whl", hash = "sha256:653d7fb2df65efefbcbf81ef5fe5e5be931f1ee4332c2893ca638c9b11a409c4"},
+    {file = "Pillow-9.4.0-cp37-cp37m-manylinux_2_28_x86_64.whl", hash = "sha256:46f39cab8bbf4a384ba7cb0bc8bae7b7062b6a11cfac1ca4bc144dea90d4a9f5"},
+    {file = "Pillow-9.4.0-cp37-cp37m-win32.whl", hash = "sha256:7ac7594397698f77bce84382929747130765f66406dc2cd8b4ab4da68ade4c6e"},
+    {file = "Pillow-9.4.0-cp37-cp37m-win_amd64.whl", hash = "sha256:46c259e87199041583658457372a183636ae8cd56dbf3f0755e0f376a7f9d0e6"},
+    {file = "Pillow-9.4.0-cp38-cp38-macosx_10_10_x86_64.whl", hash = "sha256:0e51f608da093e5d9038c592b5b575cadc12fd748af1479b5e858045fff955a9"},
+    {file = "Pillow-9.4.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:765cb54c0b8724a7c12c55146ae4647e0274a839fb6de7bcba841e04298e1011"},
+    {file = "Pillow-9.4.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:519e14e2c49fcf7616d6d2cfc5c70adae95682ae20f0395e9280db85e8d6c4df"},
+    {file = "Pillow-9.4.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d197df5489004db87d90b918033edbeee0bd6df3848a204bca3ff0a903bef837"},
+    {file = "Pillow-9.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0845adc64fe9886db00f5ab68c4a8cd933ab749a87747555cec1c95acea64b0b"},
+    {file = "Pillow-9.4.0-cp38-cp38-manylinux_2_28_aarch64.whl", hash = "sha256:e1339790c083c5a4de48f688b4841f18df839eb3c9584a770cbd818b33e26d5d"},
+    {file = "Pillow-9.4.0-cp38-cp38-manylinux_2_28_x86_64.whl", hash = "sha256:a96e6e23f2b79433390273eaf8cc94fec9c6370842e577ab10dabdcc7ea0a66b"},
+    {file = "Pillow-9.4.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:7cfc287da09f9d2a7ec146ee4d72d6ea1342e770d975e49a8621bf54eaa8f30f"},
+    {file = "Pillow-9.4.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:d7081c084ceb58278dd3cf81f836bc818978c0ccc770cbbb202125ddabec6628"},
+    {file = "Pillow-9.4.0-cp38-cp38-win32.whl", hash = "sha256:df41112ccce5d47770a0c13651479fbcd8793f34232a2dd9faeccb75eb5d0d0d"},
+    {file = "Pillow-9.4.0-cp38-cp38-win_amd64.whl", hash = "sha256:7a21222644ab69ddd9967cfe6f2bb420b460dae4289c9d40ff9a4896e7c35c9a"},
+    {file = "Pillow-9.4.0-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:0f3269304c1a7ce82f1759c12ce731ef9b6e95b6df829dccd9fe42912cc48569"},
+    {file = "Pillow-9.4.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:cb362e3b0976dc994857391b776ddaa8c13c28a16f80ac6522c23d5257156bed"},
+    {file = "Pillow-9.4.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a2e0f87144fcbbe54297cae708c5e7f9da21a4646523456b00cc956bd4c65815"},
+    {file = "Pillow-9.4.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:28676836c7796805914b76b1837a40f76827ee0d5398f72f7dcc634bae7c6264"},
+    {file = "Pillow-9.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0884ba7b515163a1a05440a138adeb722b8a6ae2c2b33aea93ea3118dd3a899e"},
+    {file = "Pillow-9.4.0-cp39-cp39-manylinux_2_28_aarch64.whl", hash = "sha256:53dcb50fbdc3fb2c55431a9b30caeb2f7027fcd2aeb501459464f0214200a503"},
+    {file = "Pillow-9.4.0-cp39-cp39-manylinux_2_28_x86_64.whl", hash = "sha256:e8c5cf126889a4de385c02a2c3d3aba4b00f70234bfddae82a5eaa3ee6d5e3e6"},
+    {file = "Pillow-9.4.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:6c6b1389ed66cdd174d040105123a5a1bc91d0aa7059c7261d20e583b6d8cbd2"},
+    {file = "Pillow-9.4.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:0dd4c681b82214b36273c18ca7ee87065a50e013112eea7d78c7a1b89a739153"},
+    {file = "Pillow-9.4.0-cp39-cp39-win32.whl", hash = "sha256:6d9dfb9959a3b0039ee06c1a1a90dc23bac3b430842dcb97908ddde05870601c"},
+    {file = "Pillow-9.4.0-cp39-cp39-win_amd64.whl", hash = "sha256:54614444887e0d3043557d9dbc697dbb16cfb5a35d672b7a0fcc1ed0cf1c600b"},
+    {file = "Pillow-9.4.0-pp38-pypy38_pp73-macosx_10_10_x86_64.whl", hash = "sha256:b9b752ab91e78234941e44abdecc07f1f0d8f51fb62941d32995b8161f68cfe5"},
+    {file = "Pillow-9.4.0-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d3b56206244dc8711f7e8b7d6cad4663917cd5b2d950799425076681e8766286"},
+    {file = "Pillow-9.4.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aabdab8ec1e7ca7f1434d042bf8b1e92056245fb179790dc97ed040361f16bfd"},
+    {file = "Pillow-9.4.0-pp38-pypy38_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:db74f5562c09953b2c5f8ec4b7dfd3f5421f31811e97d1dbc0a7c93d6e3a24df"},
+    {file = "Pillow-9.4.0-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:e9d7747847c53a16a729b6ee5e737cf170f7a16611c143d95aa60a109a59c336"},
+    {file = "Pillow-9.4.0-pp39-pypy39_pp73-macosx_10_10_x86_64.whl", hash = "sha256:b52ff4f4e002f828ea6483faf4c4e8deea8d743cf801b74910243c58acc6eda3"},
+    {file = "Pillow-9.4.0-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:575d8912dca808edd9acd6f7795199332696d3469665ef26163cd090fa1f8bfa"},
+    {file = "Pillow-9.4.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c3c4ed2ff6760e98d262e0cc9c9a7f7b8a9f61aa4d47c58835cdaf7b0b8811bb"},
+    {file = "Pillow-9.4.0-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:e621b0246192d3b9cb1dc62c78cfa4c6f6d2ddc0ec207d43c0dedecb914f152a"},
+    {file = "Pillow-9.4.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:8f127e7b028900421cad64f51f75c051b628db17fb00e099eb148761eed598c9"},
+    {file = "Pillow-9.4.0.tar.gz", hash = "sha256:a1c2d7780448eb93fbcc3789bf3916aa5720d942e37945f4056680317f1cd23e"},
 ]
 
 [package.extras]
-docs = ["furo", "olefile", "sphinx (>=2.4)", "sphinx-copybutton", "sphinx-issues (>=3.0.1)", "sphinx-removed-in", "sphinxext-opengraph"]
+docs = ["furo", "olefile", "sphinx (>=2.4)", "sphinx-copybutton", "sphinx-inline-tabs", "sphinx-issues (>=3.0.1)", "sphinx-removed-in", "sphinxext-opengraph"]
 tests = ["check-manifest", "coverage", "defusedxml", "markdown2", "olefile", "packaging", "pyroma", "pytest", "pytest-cov", "pytest-timeout"]
 
 [package.source]
@@ -1924,34 +2106,37 @@ reference = "tsinghua"
 
 [[package]]
 name = "protobuf"
-version = "3.20.3"
+version = "3.19.6"
 description = "Protocol Buffers"
 category = "main"
 optional = false
-python-versions = ">=3.7"
+python-versions = ">=3.5"
 files = [
-    {file = "protobuf-3.20.3-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:f4bd856d702e5b0d96a00ec6b307b0f51c1982c2bf9c0052cf9019e9a544ba99"},
-    {file = "protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:9aae4406ea63d825636cc11ffb34ad3379335803216ee3a856787bcf5ccc751e"},
-    {file = "protobuf-3.20.3-cp310-cp310-win32.whl", hash = "sha256:28545383d61f55b57cf4df63eebd9827754fd2dc25f80c5253f9184235db242c"},
-    {file = "protobuf-3.20.3-cp310-cp310-win_amd64.whl", hash = "sha256:67a3598f0a2dcbc58d02dd1928544e7d88f764b47d4a286202913f0b2801c2e7"},
-    {file = "protobuf-3.20.3-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:899dc660cd599d7352d6f10d83c95df430a38b410c1b66b407a6b29265d66469"},
-    {file = "protobuf-3.20.3-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:e64857f395505ebf3d2569935506ae0dfc4a15cb80dc25261176c784662cdcc4"},
-    {file = "protobuf-3.20.3-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:d9e4432ff660d67d775c66ac42a67cf2453c27cb4d738fc22cb53b5d84c135d4"},
-    {file = "protobuf-3.20.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:74480f79a023f90dc6e18febbf7b8bac7508420f2006fabd512013c0c238f454"},
-    {file = "protobuf-3.20.3-cp37-cp37m-win32.whl", hash = "sha256:b6cc7ba72a8850621bfec987cb72623e703b7fe2b9127a161ce61e61558ad905"},
-    {file = "protobuf-3.20.3-cp37-cp37m-win_amd64.whl", hash = "sha256:8c0c984a1b8fef4086329ff8dd19ac77576b384079247c770f29cc8ce3afa06c"},
-    {file = "protobuf-3.20.3-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:de78575669dddf6099a8a0f46a27e82a1783c557ccc38ee620ed8cc96d3be7d7"},
-    {file = "protobuf-3.20.3-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:f4c42102bc82a51108e449cbb32b19b180022941c727bac0cfd50170341f16ee"},
-    {file = "protobuf-3.20.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:44246bab5dd4b7fbd3c0c80b6f16686808fab0e4aca819ade6e8d294a29c7050"},
-    {file = "protobuf-3.20.3-cp38-cp38-win32.whl", hash = "sha256:c02ce36ec760252242a33967d51c289fd0e1c0e6e5cc9397e2279177716add86"},
-    {file = "protobuf-3.20.3-cp38-cp38-win_amd64.whl", hash = "sha256:447d43819997825d4e71bf5769d869b968ce96848b6479397e29fc24c4a5dfe9"},
-    {file = "protobuf-3.20.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:398a9e0c3eaceb34ec1aee71894ca3299605fa8e761544934378bbc6c97de23b"},
-    {file = "protobuf-3.20.3-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:bf01b5720be110540be4286e791db73f84a2b721072a3711efff6c324cdf074b"},
-    {file = "protobuf-3.20.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:daa564862dd0d39c00f8086f88700fdbe8bc717e993a21e90711acfed02f2402"},
-    {file = "protobuf-3.20.3-cp39-cp39-win32.whl", hash = "sha256:819559cafa1a373b7096a482b504ae8a857c89593cf3a25af743ac9ecbd23480"},
-    {file = "protobuf-3.20.3-cp39-cp39-win_amd64.whl", hash = "sha256:03038ac1cfbc41aa21f6afcbcd357281d7521b4157926f30ebecc8d4ea59dcb7"},
-    {file = "protobuf-3.20.3-py2.py3-none-any.whl", hash = "sha256:a7ca6d488aa8ff7f329d4c545b2dbad8ac31464f1d8b1c87ad1346717731e4db"},
-    {file = "protobuf-3.20.3.tar.gz", hash = "sha256:2e3427429c9cffebf259491be0af70189607f365c2f41c7c3764af6f337105f2"},
+    {file = "protobuf-3.19.6-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:010be24d5a44be7b0613750ab40bc8b8cedc796db468eae6c779b395f50d1fa1"},
+    {file = "protobuf-3.19.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:11478547958c2dfea921920617eb457bc26867b0d1aa065ab05f35080c5d9eb6"},
+    {file = "protobuf-3.19.6-cp310-cp310-win32.whl", hash = "sha256:559670e006e3173308c9254d63facb2c03865818f22204037ab76f7a0ff70b5f"},
+    {file = "protobuf-3.19.6-cp310-cp310-win_amd64.whl", hash = "sha256:347b393d4dd06fb93a77620781e11c058b3b0a5289262f094379ada2920a3730"},
+    {file = "protobuf-3.19.6-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:a8ce5ae0de28b51dff886fb922012dad885e66176663950cb2344c0439ecb473"},
+    {file = "protobuf-3.19.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:90b0d02163c4e67279ddb6dc25e063db0130fc299aefabb5d481053509fae5c8"},
+    {file = "protobuf-3.19.6-cp36-cp36m-win32.whl", hash = "sha256:30f5370d50295b246eaa0296533403961f7e64b03ea12265d6dfce3a391d8992"},
+    {file = "protobuf-3.19.6-cp36-cp36m-win_amd64.whl", hash = "sha256:0c0714b025ec057b5a7600cb66ce7c693815f897cfda6d6efb58201c472e3437"},
+    {file = "protobuf-3.19.6-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:5057c64052a1f1dd7d4450e9aac25af6bf36cfbfb3a1cd89d16393a036c49157"},
+    {file = "protobuf-3.19.6-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:bb6776bd18f01ffe9920e78e03a8676530a5d6c5911934c6a1ac6eb78973ecb6"},
+    {file = "protobuf-3.19.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:84a04134866861b11556a82dd91ea6daf1f4925746b992f277b84013a7cc1229"},
+    {file = "protobuf-3.19.6-cp37-cp37m-win32.whl", hash = "sha256:4bc98de3cdccfb5cd769620d5785b92c662b6bfad03a202b83799b6ed3fa1fa7"},
+    {file = "protobuf-3.19.6-cp37-cp37m-win_amd64.whl", hash = "sha256:aa3b82ca1f24ab5326dcf4ea00fcbda703e986b22f3d27541654f749564d778b"},
+    {file = "protobuf-3.19.6-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:2b2d2913bcda0e0ec9a784d194bc490f5dc3d9d71d322d070b11a0ade32ff6ba"},
+    {file = "protobuf-3.19.6-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:d0b635cefebd7a8a0f92020562dead912f81f401af7e71f16bf9506ff3bdbb38"},
+    {file = "protobuf-3.19.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7a552af4dc34793803f4e735aabe97ffc45962dfd3a237bdde242bff5a3de684"},
+    {file = "protobuf-3.19.6-cp38-cp38-win32.whl", hash = "sha256:0469bc66160180165e4e29de7f445e57a34ab68f49357392c5b2f54c656ab25e"},
+    {file = "protobuf-3.19.6-cp38-cp38-win_amd64.whl", hash = "sha256:91d5f1e139ff92c37e0ff07f391101df77e55ebb97f46bbc1535298d72019462"},
+    {file = "protobuf-3.19.6-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:c0ccd3f940fe7f3b35a261b1dd1b4fc850c8fde9f74207015431f174be5976b3"},
+    {file = "protobuf-3.19.6-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:30a15015d86b9c3b8d6bf78d5b8c7749f2512c29f168ca259c9d7727604d0e39"},
+    {file = "protobuf-3.19.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:878b4cd080a21ddda6ac6d1e163403ec6eea2e206cf225982ae04567d39be7b0"},
+    {file = "protobuf-3.19.6-cp39-cp39-win32.whl", hash = "sha256:5a0d7539a1b1fb7e76bf5faa0b44b30f812758e989e59c40f77a7dab320e79b9"},
+    {file = "protobuf-3.19.6-cp39-cp39-win_amd64.whl", hash = "sha256:bbf5cea5048272e1c60d235c7bd12ce1b14b8a16e76917f371c718bd3005f045"},
+    {file = "protobuf-3.19.6-py2.py3-none-any.whl", hash = "sha256:14082457dc02be946f60b15aad35e9f5c69e738f80ebbc0900a19bc83734a5a4"},
+    {file = "protobuf-3.19.6.tar.gz", hash = "sha256:5f5540d57a43042389e87661c6eaa50f47c19c6176e8cf1c4f287aeefeccb5c4"},
 ]
 
 [package.source]
@@ -2715,6 +2900,117 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "tensorflow"
+version = "2.11.0"
+description = "TensorFlow is an open source machine learning framework for everyone."
+category = "main"
+optional = true
+python-versions = ">=3.7"
+files = [
+    {file = "tensorflow-2.11.0-cp310-cp310-macosx_10_14_x86_64.whl", hash = "sha256:6c049fec6c2040685d6f43a63e17ccc5d6b0abc16b70cc6f5e7d691262b5d2d0"},
+    {file = "tensorflow-2.11.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bcc8380820cea8f68f6c90b8aee5432e8537e5bb9ec79ac61a98e6a9a02c7d40"},
+    {file = "tensorflow-2.11.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d973458241c8771bf95d4ba68ad5d67b094f72dd181c2d562ffab538c1b0dad7"},
+    {file = "tensorflow-2.11.0-cp310-cp310-win_amd64.whl", hash = "sha256:d470b772ee3c291a8c7be2331e7c379e0c338223c0bf532f5906d4556f17580d"},
+    {file = "tensorflow-2.11.0-cp37-cp37m-macosx_10_14_x86_64.whl", hash = "sha256:d29c1179149fa469ad68234c52c83081d037ead243f90e826074e2563a0f938a"},
+    {file = "tensorflow-2.11.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2cdba2fce00d6c924470d4fb65d5e95a4b6571a863860608c0c13f0393f4ca0d"},
+    {file = "tensorflow-2.11.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4f2ab20f93d2b52a44b414ec6dcf82aa12110e90e0920039a27108de28ae2728"},
+    {file = "tensorflow-2.11.0-cp37-cp37m-win_amd64.whl", hash = "sha256:445510f092f7827e1f60f59b8bfb58e664aaf05d07daaa21c5735a7f76ca2b25"},
+    {file = "tensorflow-2.11.0-cp38-cp38-macosx_10_14_x86_64.whl", hash = "sha256:056d29f2212342536ce3856aa47910a2515eb97ec0a6cc29ed47fc4be1369ec8"},
+    {file = "tensorflow-2.11.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:17b29d6d360fad545ab1127db52592efd3f19ac55c1a45e5014da328ae867ab4"},
+    {file = "tensorflow-2.11.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:335ab5cccd7a1c46e3d89d9d46913f0715e8032df8d7438f9743b3fb97b39f69"},
+    {file = "tensorflow-2.11.0-cp38-cp38-win_amd64.whl", hash = "sha256:d48da37c8ae711eb38047a56a052ca8bb4ee018a91a479e42b7a8d117628c32e"},
+    {file = "tensorflow-2.11.0-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:d9cf25bca641f2e5c77caa3bfd8dd6b892a7aec0695c54d2a7c9f52a54a8d487"},
+    {file = "tensorflow-2.11.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0d28f9691ebc48c0075e271023b3f147ae2bc29a3d3a7f42d45019c6b4a700d2"},
+    {file = "tensorflow-2.11.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:276a44210d956701899dc78ad0aa116a0071f22fb0bcc1ea6bb59f7646b08d11"},
+    {file = "tensorflow-2.11.0-cp39-cp39-win_amd64.whl", hash = "sha256:cc3444fe1d58c65a195a69656bf56015bf19dc2916da607d784b0a1e215ec008"},
+]
+
+[package.dependencies]
+absl-py = ">=1.0.0"
+astunparse = ">=1.6.0"
+flatbuffers = ">=2.0"
+gast = ">=0.2.1,<=0.4.0"
+google-pasta = ">=0.1.1"
+grpcio = ">=1.24.3,<2.0"
+h5py = ">=2.9.0"
+keras = ">=2.11.0,<2.12"
+libclang = ">=13.0.0"
+numpy = ">=1.20"
+opt-einsum = ">=2.3.2"
+packaging = "*"
+protobuf = ">=3.9.2,<3.20"
+setuptools = "*"
+six = ">=1.12.0"
+tensorboard = ">=2.11,<2.12"
+tensorflow-estimator = ">=2.11.0,<2.12"
+tensorflow-io-gcs-filesystem = {version = ">=0.23.1", markers = "platform_machine != \"arm64\" or platform_system != \"Darwin\""}
+termcolor = ">=1.1.0"
+typing-extensions = ">=3.6.6"
+wrapt = ">=1.11.0"
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
+[[package]]
+name = "tensorflow-estimator"
+version = "2.11.0"
+description = "TensorFlow Estimator."
+category = "main"
+optional = true
+python-versions = ">=3.7"
+files = [
+    {file = "tensorflow_estimator-2.11.0-py2.py3-none-any.whl", hash = "sha256:ea3b64acfff3d9a244f06178c9bdedcbdd3f125b67d0888dba8229498d06468b"},
+]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
+[[package]]
+name = "tensorflow-io-gcs-filesystem"
+version = "0.29.0"
+description = "TensorFlow IO"
+category = "main"
+optional = true
+python-versions = ">=3.7, <3.12"
+files = [
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp310-cp310-macosx_10_14_x86_64.whl", hash = "sha256:d8eb242b118721c8a23d598af69c25ad450c1e18bd1cd7ef7e8274ae0e7781ca"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:c826154e0c6bd9a1a0395a1b985976ac255a3d79d147d3eb10343b6d15710267"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9db8dc3f2c4ddfdf02f33a492600be35d0bca085aa12121a5feef173e6b5914e"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp310-cp310-win_amd64.whl", hash = "sha256:7ff4b18f1a74e1a56603fa204cf82b1af5b24ad18c579692c487b4fb4a2baec8"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp311-cp311-macosx_10_14_x86_64.whl", hash = "sha256:571fc6ba4960f3a749a362d487b60e248eb43f0abcfb0ace4f04ddb91ae04faf"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp311-cp311-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:a6297b68677a17ce388594fcf76c70718b837dba59e858280898521a858a8e4c"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp311-cp311-win_amd64.whl", hash = "sha256:ff107ac28d56bdd4c50ac69b18cc237d3a9342be8c2d11e458e19a0fac31fb9d"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp37-cp37m-macosx_10_14_x86_64.whl", hash = "sha256:6617054b8ac75cf2f19ec24ddf3a696a500758d1f755b847f3ba42aec6ad7b9e"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:ec87a475a4024bc8c4427c6cbeba009fd76b1b95ad764755fdf958c234470acd"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:931e225b406d67d0a8a8f549242a0a1d173a02e0179d86b88351de19c393e06f"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp37-cp37m-win_amd64.whl", hash = "sha256:975b674d7816c08cf47a50cfc5b73a36ca0b3ef32d8e37a788b7cae38b9e1549"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp38-cp38-macosx_10_14_x86_64.whl", hash = "sha256:e114d672fc565985468d6a26a1e54e2f0002ab76c711f49a56d948ad05718c80"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:630372fe2f1c05b57c4ad0c9050b570bc67e474ec513cf046832f8889002fab7"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d0736853c35030e027c0f9577f07355e5cd21463321008ef4ee8b390f913fdd6"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp38-cp38-win_amd64.whl", hash = "sha256:049b21d5056f57952b0450c2bac9c2bf2fabc4bbb571470abf0cba4243a41dfe"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:655af38843d83ef322db190d162e21060ca143935a04f4b64b29f60c1c4dc073"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:8543e14b2f32771e7a7eca7a3d34a8fbdf1f4e9ae7d346bcacff011f43c693cb"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ef5656932c1f4d4e4d366cdae469562344ecd38cd51de70ebf60e68ee0834da1"},
+    {file = "tensorflow_io_gcs_filesystem-0.29.0-cp39-cp39-win_amd64.whl", hash = "sha256:bcb7405474ed136b3a2e2181a732968c52ba9f35a03fe296acc9f1ec4e7044ee"},
+]
+
+[package.extras]
+tensorflow = ["tensorflow (>=2.11.0,<2.12.0)"]
+tensorflow-aarch64 = ["tensorflow-aarch64 (>=2.11.0,<2.12.0)"]
+tensorflow-cpu = ["tensorflow-cpu (>=2.11.0,<2.12.0)"]
+tensorflow-gpu = ["tensorflow-gpu (>=2.11.0,<2.12.0)"]
+tensorflow-rocm = ["tensorflow-rocm (>=2.11.0,<2.12.0)"]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "termcolor"
 version = "2.1.1"
@@ -3025,6 +3321,85 @@ type = "legacy"
 url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
+[[package]]
+name = "wrapt"
+version = "1.14.1"
+description = "Module for decorators, wrappers and monkey patching."
+category = "main"
+optional = true
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7"
+files = [
+    {file = "wrapt-1.14.1-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:1b376b3f4896e7930f1f772ac4b064ac12598d1c38d04907e696cc4d794b43d3"},
+    {file = "wrapt-1.14.1-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:903500616422a40a98a5a3c4ff4ed9d0066f3b4c951fa286018ecdf0750194ef"},
+    {file = "wrapt-1.14.1-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:5a9a0d155deafd9448baff28c08e150d9b24ff010e899311ddd63c45c2445e28"},
+    {file = "wrapt-1.14.1-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:ddaea91abf8b0d13443f6dac52e89051a5063c7d014710dcb4d4abb2ff811a59"},
+    {file = "wrapt-1.14.1-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:36f582d0c6bc99d5f39cd3ac2a9062e57f3cf606ade29a0a0d6b323462f4dd87"},
+    {file = "wrapt-1.14.1-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:7ef58fb89674095bfc57c4069e95d7a31cfdc0939e2a579882ac7d55aadfd2a1"},
+    {file = "wrapt-1.14.1-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:e2f83e18fe2f4c9e7db597e988f72712c0c3676d337d8b101f6758107c42425b"},
+    {file = "wrapt-1.14.1-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:ee2b1b1769f6707a8a445162ea16dddf74285c3964f605877a20e38545c3c462"},
+    {file = "wrapt-1.14.1-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:833b58d5d0b7e5b9832869f039203389ac7cbf01765639c7309fd50ef619e0b1"},
+    {file = "wrapt-1.14.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:80bb5c256f1415f747011dc3604b59bc1f91c6e7150bd7db03b19170ee06b320"},
+    {file = "wrapt-1.14.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:07f7a7d0f388028b2df1d916e94bbb40624c59b48ecc6cbc232546706fac74c2"},
+    {file = "wrapt-1.14.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:02b41b633c6261feff8ddd8d11c711df6842aba629fdd3da10249a53211a72c4"},
+    {file = "wrapt-1.14.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2fe803deacd09a233e4762a1adcea5db5d31e6be577a43352936179d14d90069"},
+    {file = "wrapt-1.14.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:257fd78c513e0fb5cdbe058c27a0624c9884e735bbd131935fd49e9fe719d310"},
+    {file = "wrapt-1.14.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:4fcc4649dc762cddacd193e6b55bc02edca674067f5f98166d7713b193932b7f"},
+    {file = "wrapt-1.14.1-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:11871514607b15cfeb87c547a49bca19fde402f32e2b1c24a632506c0a756656"},
+    {file = "wrapt-1.14.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:8ad85f7f4e20964db4daadcab70b47ab05c7c1cf2a7c1e51087bfaa83831854c"},
+    {file = "wrapt-1.14.1-cp310-cp310-win32.whl", hash = "sha256:a9a52172be0b5aae932bef82a79ec0a0ce87288c7d132946d645eba03f0ad8a8"},
+    {file = "wrapt-1.14.1-cp310-cp310-win_amd64.whl", hash = "sha256:6d323e1554b3d22cfc03cd3243b5bb815a51f5249fdcbb86fda4bf62bab9e164"},
+    {file = "wrapt-1.14.1-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:43ca3bbbe97af00f49efb06e352eae40434ca9d915906f77def219b88e85d907"},
+    {file = "wrapt-1.14.1-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:6b1a564e6cb69922c7fe3a678b9f9a3c54e72b469875aa8018f18b4d1dd1adf3"},
+    {file = "wrapt-1.14.1-cp35-cp35m-manylinux2010_i686.whl", hash = "sha256:00b6d4ea20a906c0ca56d84f93065b398ab74b927a7a3dbd470f6fc503f95dc3"},
+    {file = "wrapt-1.14.1-cp35-cp35m-manylinux2010_x86_64.whl", hash = "sha256:a85d2b46be66a71bedde836d9e41859879cc54a2a04fad1191eb50c2066f6e9d"},
+    {file = "wrapt-1.14.1-cp35-cp35m-win32.whl", hash = "sha256:dbcda74c67263139358f4d188ae5faae95c30929281bc6866d00573783c422b7"},
+    {file = "wrapt-1.14.1-cp35-cp35m-win_amd64.whl", hash = "sha256:b21bb4c09ffabfa0e85e3a6b623e19b80e7acd709b9f91452b8297ace2a8ab00"},
+    {file = "wrapt-1.14.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:9e0fd32e0148dd5dea6af5fee42beb949098564cc23211a88d799e434255a1f4"},
+    {file = "wrapt-1.14.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9736af4641846491aedb3c3f56b9bc5568d92b0692303b5a305301a95dfd38b1"},
+    {file = "wrapt-1.14.1-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:5b02d65b9ccf0ef6c34cba6cf5bf2aab1bb2f49c6090bafeecc9cd81ad4ea1c1"},
+    {file = "wrapt-1.14.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:21ac0156c4b089b330b7666db40feee30a5d52634cc4560e1905d6529a3897ff"},
+    {file = "wrapt-1.14.1-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:9f3e6f9e05148ff90002b884fbc2a86bd303ae847e472f44ecc06c2cd2fcdb2d"},
+    {file = "wrapt-1.14.1-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:6e743de5e9c3d1b7185870f480587b75b1cb604832e380d64f9504a0535912d1"},
+    {file = "wrapt-1.14.1-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:d79d7d5dc8a32b7093e81e97dad755127ff77bcc899e845f41bf71747af0c569"},
+    {file = "wrapt-1.14.1-cp36-cp36m-win32.whl", hash = "sha256:81b19725065dcb43df02b37e03278c011a09e49757287dca60c5aecdd5a0b8ed"},
+    {file = "wrapt-1.14.1-cp36-cp36m-win_amd64.whl", hash = "sha256:b014c23646a467558be7da3d6b9fa409b2c567d2110599b7cf9a0c5992b3b471"},
+    {file = "wrapt-1.14.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:88bd7b6bd70a5b6803c1abf6bca012f7ed963e58c68d76ee20b9d751c74a3248"},
+    {file = "wrapt-1.14.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b5901a312f4d14c59918c221323068fad0540e34324925c8475263841dbdfe68"},
+    {file = "wrapt-1.14.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d77c85fedff92cf788face9bfa3ebaa364448ebb1d765302e9af11bf449ca36d"},
+    {file = "wrapt-1.14.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8d649d616e5c6a678b26d15ece345354f7c2286acd6db868e65fcc5ff7c24a77"},
+    {file = "wrapt-1.14.1-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:7d2872609603cb35ca513d7404a94d6d608fc13211563571117046c9d2bcc3d7"},
+    {file = "wrapt-1.14.1-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:ee6acae74a2b91865910eef5e7de37dc6895ad96fa23603d1d27ea69df545015"},
+    {file = "wrapt-1.14.1-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:2b39d38039a1fdad98c87279b48bc5dce2c0ca0d73483b12cb72aa9609278e8a"},
+    {file = "wrapt-1.14.1-cp37-cp37m-win32.whl", hash = "sha256:60db23fa423575eeb65ea430cee741acb7c26a1365d103f7b0f6ec412b893853"},
+    {file = "wrapt-1.14.1-cp37-cp37m-win_amd64.whl", hash = "sha256:709fe01086a55cf79d20f741f39325018f4df051ef39fe921b1ebe780a66184c"},
+    {file = "wrapt-1.14.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:8c0ce1e99116d5ab21355d8ebe53d9460366704ea38ae4d9f6933188f327b456"},
+    {file = "wrapt-1.14.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:e3fb1677c720409d5f671e39bac6c9e0e422584e5f518bfd50aa4cbbea02433f"},
+    {file = "wrapt-1.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:642c2e7a804fcf18c222e1060df25fc210b9c58db7c91416fb055897fc27e8cc"},
+    {file = "wrapt-1.14.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:7b7c050ae976e286906dd3f26009e117eb000fb2cf3533398c5ad9ccc86867b1"},
+    {file = "wrapt-1.14.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ef3f72c9666bba2bab70d2a8b79f2c6d2c1a42a7f7e2b0ec83bb2f9e383950af"},
+    {file = "wrapt-1.14.1-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:01c205616a89d09827986bc4e859bcabd64f5a0662a7fe95e0d359424e0e071b"},
+    {file = "wrapt-1.14.1-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:5a0f54ce2c092aaf439813735584b9537cad479575a09892b8352fea5e988dc0"},
+    {file = "wrapt-1.14.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:2cf71233a0ed05ccdabe209c606fe0bac7379fdcf687f39b944420d2a09fdb57"},
+    {file = "wrapt-1.14.1-cp38-cp38-win32.whl", hash = "sha256:aa31fdcc33fef9eb2552cbcbfee7773d5a6792c137b359e82879c101e98584c5"},
+    {file = "wrapt-1.14.1-cp38-cp38-win_amd64.whl", hash = "sha256:d1967f46ea8f2db647c786e78d8cc7e4313dbd1b0aca360592d8027b8508e24d"},
+    {file = "wrapt-1.14.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:3232822c7d98d23895ccc443bbdf57c7412c5a65996c30442ebe6ed3df335383"},
+    {file = "wrapt-1.14.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:988635d122aaf2bdcef9e795435662bcd65b02f4f4c1ae37fbee7401c440b3a7"},
+    {file = "wrapt-1.14.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9cca3c2cdadb362116235fdbd411735de4328c61425b0aa9f872fd76d02c4e86"},
+    {file = "wrapt-1.14.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d52a25136894c63de15a35bc0bdc5adb4b0e173b9c0d07a2be9d3ca64a332735"},
+    {file = "wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:40e7bc81c9e2b2734ea4bc1aceb8a8f0ceaac7c5299bc5d69e37c44d9081d43b"},
+    {file = "wrapt-1.14.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:b9b7a708dd92306328117d8c4b62e2194d00c365f18eff11a9b53c6f923b01e3"},
+    {file = "wrapt-1.14.1-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:6a9a25751acb379b466ff6be78a315e2b439d4c94c1e99cb7266d40a537995d3"},
+    {file = "wrapt-1.14.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:34aa51c45f28ba7f12accd624225e2b1e5a3a45206aa191f6f9aac931d9d56fe"},
+    {file = "wrapt-1.14.1-cp39-cp39-win32.whl", hash = "sha256:dee0ce50c6a2dd9056c20db781e9c1cfd33e77d2d569f5d1d9321c641bb903d5"},
+    {file = "wrapt-1.14.1-cp39-cp39-win_amd64.whl", hash = "sha256:dee60e1de1898bde3b238f18340eec6148986da0455d8ba7848d50470a7a32fb"},
+    {file = "wrapt-1.14.1.tar.gz", hash = "sha256:380a85cf89e0e69b7cfbe2ea9f765f004ff419f34194018a6827ac0e3edfed4d"},
+]
+
+[package.source]
+type = "legacy"
+url = "https://pypi.tuna.tsinghua.edu.cn/simple"
+reference = "tsinghua"
+
 [[package]]
 name = "zipp"
 version = "3.11.0"
@@ -3047,12 +3422,13 @@ url = "https://pypi.tuna.tsinghua.edu.cn/simple"
 reference = "tsinghua"
 
 [extras]
-all = ["torch", "shimmy", "autorom", "mujoco", "mujoco-py", "imageio", "free-mujoco-py"]
+all = ["torch", "tensorflow", "shimmy", "autorom", "mujoco", "mujoco-py", "imageio", "free-mujoco-py"]
 atari = ["shimmy", "autorom"]
 mujoco = ["mujoco", "mujoco-py", "imageio", "free-mujoco-py"]
+tf = ["tensorflow"]
 torch = ["torch"]
 
 [metadata]
 lock-version = "2.0"
 python-versions = ">=3.8,<3.11"
-content-hash = "ed78338cd92286ac78649646c3ce06a54abdfe91fa0097897556aeb5b88f34fe"
+content-hash = "08145dceccb2397e9e36c77e6e109b186589db08c1db12c0941f90dd75e8a3a3"
diff --git a/pyproject.toml b/pyproject.toml
index 26db16b..704a5e4 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -48,9 +48,12 @@ fire = "^0.5.0"
 # torch
 torch = { version = "^1.12.1", optional = true }
 
+# tensorflow2
+tensorflow = { version = "^2.11.0", optional = true }
+
 # atari
 shimmy = { version = ">=0.1.0,<1.0", extras = ["atari"], optional = true }
-autorom = { version = "~=0.4.2", extras = ["accept-rom-license"], optional = true }
+autorom = { version = ">=0.4.2,<0.6.0", extras = ["accept-rom-license"], optional = true }
 
 # mujoco
 mujoco = { version = ">=2.3.1.post1", optional = true }
@@ -60,9 +63,10 @@ free-mujoco-py = { version = "2.1.6", optional = true }
 
 [tool.poetry.extras]
 torch = ["torch"]
+tf = ["tensorflow"]
 atari = ["shimmy", "autorom"]
 mujoco = ["mujoco", "mujoco-py", "imageio", "free-mujoco-py"]
-all = ["torch", "shimmy", "autorom", "mujoco", "mujoco-py", "imageio", "free-mujoco-py"]
+all = ["torch", "tensorflow", "shimmy", "autorom", "mujoco", "mujoco-py", "imageio", "free-mujoco-py"]
 
 [tool.poetry.group.dev]
 optional = true
diff --git a/requirements/requirements-atari-tf-dev.txt b/requirements/requirements-atari-tf-dev.txt
new file mode 100644
index 0000000..9522ad5
--- /dev/null
+++ b/requirements/requirements-atari-tf-dev.txt
@@ -0,0 +1,112 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
+autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+cfgv==3.3.1 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and sys_platform == "win32" or python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+distlib==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+exceptiongroup==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+filelock==3.9.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+ghp-import==2.1.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+identify==2.5.11 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+iniconfig==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+jinja2==3.1.2 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+libtorrent==2.0.7 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mergedeep==1.3.4 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-git-revision-date-localized-plugin==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-glightbox==0.3.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material-extensions==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material==8.5.11 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-static-i18n==0.53 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs==1.4.2 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+nodeenv==1.7.0 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
+pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
+pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pygments==2.14.0 ; python_version >= "3.8" and python_version < "3.11"
+pymdown-extensions==9.9 ; python_version >= "3.8" and python_version < "3.11"
+pytest==7.2.0 ; python_version >= "3.8" and python_version < "3.11"
+python-dateutil==2.8.2 ; python_version >= "3.8" and python_version < "3.11"
+pytz==2022.7 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml-env-tag==0.1 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+soupsieve==2.3.2.post1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tomli==2.0.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+virtualenv==20.17.1 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+watchdog==2.2.1 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-atari-tf.txt b/requirements/requirements-atari-tf.txt
new file mode 100644
index 0000000..b196e0d
--- /dev/null
+++ b/requirements/requirements-atari-tf.txt
@@ -0,0 +1,80 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+libtorrent==2.0.7 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-torch-atari-dev.txt b/requirements/requirements-atari-torch-dev.txt
similarity index 97%
rename from requirements/requirements-torch-atari-dev.txt
rename to requirements/requirements-atari-torch-dev.txt
index e1cad08..0769189 100644
--- a/requirements/requirements-torch-atari-dev.txt
+++ b/requirements/requirements-atari-torch-dev.txt
@@ -2,7 +2,7 @@ absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
 ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
 attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
 autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
-autorom[accept-rom-license]==0.4.2 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
 babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
 beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
 cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
@@ -58,13 +58,13 @@ oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
 packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
 pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
 pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
diff --git a/requirements/requirements-torch-atari.txt b/requirements/requirements-atari-torch.txt
similarity index 96%
rename from requirements/requirements-torch-atari.txt
rename to requirements/requirements-atari-torch.txt
index 828b33c..7d481fd 100644
--- a/requirements/requirements-torch-atari.txt
+++ b/requirements/requirements-atari-torch.txt
@@ -1,7 +1,7 @@
 absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
 ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
 autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
-autorom[accept-rom-license]==0.4.2 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
 cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
 certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
 charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
@@ -38,10 +38,10 @@ nvidia-cudnn-cu11==8.5.0.96 ; python_version >= "3.8" and python_version < "3.11
 oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
diff --git a/requirements/requirements-basic-tf-dev.txt b/requirements/requirements-basic-tf-dev.txt
new file mode 100644
index 0000000..13855d7
--- /dev/null
+++ b/requirements/requirements-basic-tf-dev.txt
@@ -0,0 +1,109 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
+babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+cfgv==3.3.1 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and sys_platform == "win32" or python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+distlib==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+exceptiongroup==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+filelock==3.9.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+ghp-import==2.1.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+identify==2.5.11 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+iniconfig==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+jinja2==3.1.2 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mergedeep==1.3.4 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-git-revision-date-localized-plugin==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-glightbox==0.3.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material-extensions==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material==8.5.11 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-static-i18n==0.53 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs==1.4.2 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+nodeenv==1.7.0 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
+pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
+pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pygments==2.14.0 ; python_version >= "3.8" and python_version < "3.11"
+pymdown-extensions==9.9 ; python_version >= "3.8" and python_version < "3.11"
+pytest==7.2.0 ; python_version >= "3.8" and python_version < "3.11"
+python-dateutil==2.8.2 ; python_version >= "3.8" and python_version < "3.11"
+pytz==2022.7 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml-env-tag==0.1 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+soupsieve==2.3.2.post1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tomli==2.0.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+virtualenv==20.17.1 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+watchdog==2.2.1 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-basic-tf.txt b/requirements/requirements-basic-tf.txt
new file mode 100644
index 0000000..57abcbf
--- /dev/null
+++ b/requirements/requirements-basic-tf.txt
@@ -0,0 +1,77 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-basic-torch-dev.txt b/requirements/requirements-basic-torch-dev.txt
new file mode 100644
index 0000000..ed88be6
--- /dev/null
+++ b/requirements/requirements-basic-torch-dev.txt
@@ -0,0 +1,102 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
+babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+cfgv==3.3.1 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and sys_platform == "win32" or python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+distlib==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+exceptiongroup==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+filelock==3.9.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+ghp-import==2.1.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+identify==2.5.11 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+iniconfig==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+jinja2==3.1.2 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mergedeep==1.3.4 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-git-revision-date-localized-plugin==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-glightbox==0.3.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material-extensions==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material==8.5.11 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-static-i18n==0.53 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs==1.4.2 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+nodeenv==1.7.0 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+nvidia-cublas-cu11==11.10.3.66 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+nvidia-cuda-nvrtc-cu11==11.7.99 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+nvidia-cuda-runtime-cu11==11.7.99 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+nvidia-cudnn-cu11==8.5.0.96 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
+pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
+pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pygments==2.14.0 ; python_version >= "3.8" and python_version < "3.11"
+pymdown-extensions==9.9 ; python_version >= "3.8" and python_version < "3.11"
+pytest==7.2.0 ; python_version >= "3.8" and python_version < "3.11"
+python-dateutil==2.8.2 ; python_version >= "3.8" and python_version < "3.11"
+pytz==2022.7 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml-env-tag==0.1 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+soupsieve==2.3.2.post1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tomli==2.0.1 ; python_version >= "3.8" and python_version < "3.11"
+torch==1.13.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+virtualenv==20.17.1 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+watchdog==2.2.1 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-basic-torch.txt b/requirements/requirements-basic-torch.txt
new file mode 100644
index 0000000..752c6ca
--- /dev/null
+++ b/requirements/requirements-basic-torch.txt
@@ -0,0 +1,69 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+nvidia-cublas-cu11==11.10.3.66 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+nvidia-cuda-nvrtc-cu11==11.7.99 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+nvidia-cuda-runtime-cu11==11.7.99 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+nvidia-cudnn-cu11==8.5.0.96 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+torch==1.13.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-dev.txt b/requirements/requirements-dev.txt
index 083f49b..a665f86 100644
--- a/requirements/requirements-dev.txt
+++ b/requirements/requirements-dev.txt
@@ -1,8 +1,9 @@
 absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
 ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
 attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
 autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
-autorom[accept-rom-license]==0.4.2 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
 babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
 beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
 cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
@@ -23,16 +24,20 @@ exceptiongroup==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
 fasteners==0.15 ; python_version >= "3.8" and python_version < "3.11"
 filelock==3.9.0 ; python_version >= "3.8" and python_version < "3.11"
 fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
 free-mujoco-py==2.1.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
 ghp-import==2.1.0 ; python_version >= "3.8" and python_version < "3.11"
 gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
 gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
 glfw==1.12.0 ; python_version >= "3.8" and python_version < "3.11"
 google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
 google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
 grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
 gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
 gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
 identify==2.5.11 ; python_version >= "3.8" and python_version < "3.11"
 idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
 imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
@@ -42,6 +47,8 @@ importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11
 iniconfig==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
 jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
 jinja2==3.1.2 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
 libtorrent==2.0.7 ; python_version >= "3.8" and python_version < "3.11"
 markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
 markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
@@ -64,15 +71,16 @@ nvidia-cuda-runtime-cu11==11.7.99 ; python_version >= "3.8" and python_version <
 nvidia-cudnn-cu11==8.5.0.96 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
 oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
 packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
 pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
 pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
@@ -101,6 +109,9 @@ soupsieve==2.3.2.post1 ; python_version >= "3.8" and python_version < "3.11"
 tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
 tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
 tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
 termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
 tomli==2.0.1 ; python_version >= "3.8" and python_version < "3.11"
 torch==1.13.1 ; python_version >= "3.8" and python_version < "3.11"
@@ -112,4 +123,5 @@ wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
 watchdog==2.2.1 ; python_version >= "3.8" and python_version < "3.11"
 werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
 wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
 zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-mujoco-tf-dev.txt b/requirements/requirements-mujoco-tf-dev.txt
new file mode 100644
index 0000000..9cd75eb
--- /dev/null
+++ b/requirements/requirements-mujoco-tf-dev.txt
@@ -0,0 +1,119 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
+babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+cffi==1.15.1 ; python_version >= "3.8" and python_version < "3.11"
+cfgv==3.3.1 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and sys_platform == "win32" or python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+cython==0.29.32 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+distlib==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+exceptiongroup==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+fasteners==0.15 ; python_version >= "3.8" and python_version < "3.11"
+filelock==3.9.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+free-mujoco-py==2.1.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+ghp-import==2.1.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+glfw==1.12.0 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+identify==2.5.11 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+iniconfig==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+jinja2==3.1.2 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mergedeep==1.3.4 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-git-revision-date-localized-plugin==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-glightbox==0.3.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material-extensions==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material==8.5.11 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-static-i18n==0.53 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs==1.4.2 ; python_version >= "3.8" and python_version < "3.11"
+monotonic==1.6 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+mujoco-py==2.1.2.14 ; python_version >= "3.8" and python_version < "3.11"
+mujoco==2.3.1.post1 ; python_version >= "3.8" and python_version < "3.11"
+nodeenv==1.7.0 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
+pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
+pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pycparser==2.21 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pygments==2.14.0 ; python_version >= "3.8" and python_version < "3.11"
+pymdown-extensions==9.9 ; python_version >= "3.8" and python_version < "3.11"
+pyopengl==3.1.6 ; python_version >= "3.8" and python_version < "3.11"
+pytest==7.2.0 ; python_version >= "3.8" and python_version < "3.11"
+python-dateutil==2.8.2 ; python_version >= "3.8" and python_version < "3.11"
+pytz==2022.7 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml-env-tag==0.1 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+soupsieve==2.3.2.post1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tomli==2.0.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+virtualenv==20.17.1 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+watchdog==2.2.1 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-mujoco-tf.txt b/requirements/requirements-mujoco-tf.txt
new file mode 100644
index 0000000..2e59e32
--- /dev/null
+++ b/requirements/requirements-mujoco-tf.txt
@@ -0,0 +1,87 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+cffi==1.15.1 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+cython==0.29.32 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+fasteners==0.15 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+free-mujoco-py==2.1.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+glfw==1.12.0 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+monotonic==1.6 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+mujoco-py==2.1.2.14 ; python_version >= "3.8" and python_version < "3.11"
+mujoco==2.3.1.post1 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pycparser==2.21 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pyopengl==3.1.6 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-torch-mujoco-dev.txt b/requirements/requirements-mujoco-torch-dev.txt
similarity index 98%
rename from requirements/requirements-torch-mujoco-dev.txt
rename to requirements/requirements-mujoco-torch-dev.txt
index 6e8116c..f6f27fa 100644
--- a/requirements/requirements-torch-mujoco-dev.txt
+++ b/requirements/requirements-mujoco-torch-dev.txt
@@ -63,13 +63,13 @@ oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
 packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
 pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
 pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
diff --git a/requirements/requirements-torch-mujoco.txt b/requirements/requirements-mujoco-torch.txt
similarity index 97%
rename from requirements/requirements-torch-mujoco.txt
rename to requirements/requirements-mujoco-torch.txt
index 9aac9b7..595ba17 100644
--- a/requirements/requirements-torch-mujoco.txt
+++ b/requirements/requirements-mujoco-torch.txt
@@ -43,10 +43,10 @@ nvidia-cudnn-cu11==8.5.0.96 ; python_version >= "3.8" and python_version < "3.11
 oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
diff --git a/requirements/requirements-tf-dev.txt b/requirements/requirements-tf-dev.txt
new file mode 100644
index 0000000..71bdf12
--- /dev/null
+++ b/requirements/requirements-tf-dev.txt
@@ -0,0 +1,122 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
+autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+cffi==1.15.1 ; python_version >= "3.8" and python_version < "3.11"
+cfgv==3.3.1 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and sys_platform == "win32" or python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+cython==0.29.32 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+distlib==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+exceptiongroup==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+fasteners==0.15 ; python_version >= "3.8" and python_version < "3.11"
+filelock==3.9.0 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+free-mujoco-py==2.1.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+ghp-import==2.1.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+glfw==1.12.0 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+identify==2.5.11 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+iniconfig==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+jinja2==3.1.2 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+libtorrent==2.0.7 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mergedeep==1.3.4 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-git-revision-date-localized-plugin==1.1.0 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-glightbox==0.3.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material-extensions==1.1.1 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-material==8.5.11 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs-static-i18n==0.53 ; python_version >= "3.8" and python_version < "3.11"
+mkdocs==1.4.2 ; python_version >= "3.8" and python_version < "3.11"
+monotonic==1.6 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+mujoco-py==2.1.2.14 ; python_version >= "3.8" and python_version < "3.11"
+mujoco==2.3.1.post1 ; python_version >= "3.8" and python_version < "3.11"
+nodeenv==1.7.0 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
+pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
+pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pycparser==2.21 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pygments==2.14.0 ; python_version >= "3.8" and python_version < "3.11"
+pymdown-extensions==9.9 ; python_version >= "3.8" and python_version < "3.11"
+pyopengl==3.1.6 ; python_version >= "3.8" and python_version < "3.11"
+pytest==7.2.0 ; python_version >= "3.8" and python_version < "3.11"
+python-dateutil==2.8.2 ; python_version >= "3.8" and python_version < "3.11"
+pytz==2022.7 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml-env-tag==0.1 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+soupsieve==2.3.2.post1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tomli==2.0.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+virtualenv==20.17.1 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+watchdog==2.2.1 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-tf.txt b/requirements/requirements-tf.txt
new file mode 100644
index 0000000..2b9a3d5
--- /dev/null
+++ b/requirements/requirements-tf.txt
@@ -0,0 +1,90 @@
+absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
+ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
+autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
+certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
+cffi==1.15.1 ; python_version >= "3.8" and python_version < "3.11"
+charset-normalizer==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+click==8.1.3 ; python_version >= "3.8" and python_version < "3.11"
+cloudpickle==2.2.0 ; python_version >= "3.8" and python_version < "3.11"
+colorama==0.4.6 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Windows"
+combine-signatures==0.1.0 ; python_version >= "3.8" and python_version < "3.11"
+cython==0.29.32 ; python_version >= "3.8" and python_version < "3.11"
+decorator==4.4.2 ; python_version >= "3.8" and python_version < "3.11"
+dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
+docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+fasteners==0.15 ; python_version >= "3.8" and python_version < "3.11"
+fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
+free-mujoco-py==2.1.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
+gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
+gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
+glfw==1.12.0 ; python_version >= "3.8" and python_version < "3.11"
+google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
+google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
+gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
+idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
+imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
+imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
+importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
+importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
+jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
+libtorrent==2.0.7 ; python_version >= "3.8" and python_version < "3.11"
+markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
+markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+monotonic==1.6 ; python_version >= "3.8" and python_version < "3.11"
+moviepy==1.0.3 ; python_version >= "3.8" and python_version < "3.11"
+mujoco-py==2.1.2.14 ; python_version >= "3.8" and python_version < "3.11"
+mujoco==2.3.1.post1 ; python_version >= "3.8" and python_version < "3.11"
+numpy==1.24.1 ; python_version < "3.11" and python_version >= "3.8"
+oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
+opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
+pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
+proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
+promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
+psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
+pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
+pycparser==2.21 ; python_version >= "3.8" and python_version < "3.11"
+pygame==2.1.3.dev8 ; python_version >= "3.8" and python_version < "3.11"
+pyopengl==3.1.6 ; python_version >= "3.8" and python_version < "3.11"
+pyyaml==6.0 ; python_version >= "3.8" and python_version < "3.11"
+requests-oauthlib==1.3.1 ; python_version >= "3.8" and python_version < "3.11"
+requests==2.28.1 ; python_version >= "3.8" and python_version < "3.11"
+rsa==4.9 ; python_version >= "3.8" and python_version < "3.11"
+sentry-sdk==1.12.1 ; python_version >= "3.8" and python_version < "3.11"
+setproctitle==1.3.2 ; python_version >= "3.8" and python_version < "3.11"
+setuptools==65.6.3 ; python_version >= "3.8" and python_version < "3.11"
+shimmy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shimmy[atari]==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+shortuuid==1.0.11 ; python_version >= "3.8" and python_version < "3.11"
+six==1.16.0 ; python_version >= "3.8" and python_version < "3.11"
+smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
+tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
+tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
+typing-extensions==4.4.0 ; python_version >= "3.8" and python_version < "3.11"
+urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
+wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
+werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
+wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
+zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"
diff --git a/requirements/requirements-torch-dev.txt b/requirements/requirements-torch-dev.txt
index 083f49b..5dcbd8f 100644
--- a/requirements/requirements-torch-dev.txt
+++ b/requirements/requirements-torch-dev.txt
@@ -2,7 +2,7 @@ absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
 ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
 attrs==22.2.0 ; python_version >= "3.8" and python_version < "3.11"
 autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
-autorom[accept-rom-license]==0.4.2 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
 babel==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
 beautifulsoup4==4.11.1 ; python_version >= "3.8" and python_version < "3.11"
 cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
@@ -66,13 +66,13 @@ oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
 packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 platformdirs==2.6.2 ; python_version >= "3.8" and python_version < "3.11"
 pluggy==1.0.0 ; python_version >= "3.8" and python_version < "3.11"
 pre-commit==2.21.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
diff --git a/requirements/requirements-torch.txt b/requirements/requirements-torch.txt
index e0bb9d4..c956940 100644
--- a/requirements/requirements-torch.txt
+++ b/requirements/requirements-torch.txt
@@ -1,7 +1,7 @@
 absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
 ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
 autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
-autorom[accept-rom-license]==0.4.2 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
 cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
 certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
 cffi==1.15.1 ; python_version >= "3.8" and python_version < "3.11"
@@ -46,10 +46,10 @@ nvidia-cudnn-cu11==8.5.0.96 ; python_version >= "3.8" and python_version < "3.11
 oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
diff --git a/requirements/requirements.txt b/requirements/requirements.txt
index e0bb9d4..b8ec428 100644
--- a/requirements/requirements.txt
+++ b/requirements/requirements.txt
@@ -1,7 +1,8 @@
 absl-py==1.3.0 ; python_version >= "3.8" and python_version < "3.11"
 ale-py==0.8.0 ; python_version >= "3.8" and python_version < "3.11"
+astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.11"
 autorom-accept-rom-license==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
-autorom[accept-rom-license]==0.4.2 ; python_version >= "3.8" and python_version < "3.11"
+autorom[accept-rom-license]==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
 cachetools==5.2.0 ; python_version >= "3.8" and python_version < "3.11"
 certifi==2022.12.7 ; python_version >= "3.8" and python_version < "3.11"
 cffi==1.15.1 ; python_version >= "3.8" and python_version < "3.11"
@@ -16,21 +17,27 @@ dill==0.3.6 ; python_version >= "3.8" and python_version < "3.11"
 docker-pycreds==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
 fasteners==0.15 ; python_version >= "3.8" and python_version < "3.11"
 fire==0.5.0 ; python_version >= "3.8" and python_version < "3.11"
+flatbuffers==22.12.6 ; python_version >= "3.8" and python_version < "3.11"
 free-mujoco-py==2.1.6 ; python_version >= "3.8" and python_version < "3.11"
+gast==0.4.0 ; python_version >= "3.8" and python_version < "3.11"
 gitdb==4.0.10 ; python_version >= "3.8" and python_version < "3.11"
 gitpython==3.1.30 ; python_version >= "3.8" and python_version < "3.11"
 glfw==1.12.0 ; python_version >= "3.8" and python_version < "3.11"
 google-auth-oauthlib==0.4.6 ; python_version >= "3.8" and python_version < "3.11"
 google-auth==2.15.0 ; python_version >= "3.8" and python_version < "3.11"
+google-pasta==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
 grpcio==1.51.1 ; python_version >= "3.8" and python_version < "3.11"
 gymnasium-notices==0.0.1 ; python_version >= "3.8" and python_version < "3.11"
 gymnasium==0.27.0 ; python_version >= "3.8" and python_version < "3.11"
+h5py==3.7.0 ; python_version >= "3.8" and python_version < "3.11"
 idna==3.4 ; python_version >= "3.8" and python_version < "3.11"
 imageio-ffmpeg==0.4.7 ; python_version >= "3.8" and python_version < "3.11"
 imageio==2.23.0 ; python_version >= "3.8" and python_version < "3.11"
 importlib-metadata==6.0.0 ; python_version >= "3.8" and python_version < "3.10"
 importlib-resources==5.10.2 ; python_version >= "3.8" and python_version < "3.11"
 jax-jumpy==0.2.0 ; python_version >= "3.8" and python_version < "3.11"
+keras==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+libclang==14.0.6 ; python_version >= "3.8" and python_version < "3.11"
 libtorrent==2.0.7 ; python_version >= "3.8" and python_version < "3.11"
 markdown==3.3.7 ; python_version >= "3.8" and python_version < "3.11"
 markupsafe==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
@@ -45,11 +52,13 @@ nvidia-cuda-runtime-cu11==11.7.99 ; python_version >= "3.8" and python_version <
 nvidia-cudnn-cu11==8.5.0.96 ; python_version >= "3.8" and python_version < "3.11" and platform_system == "Linux"
 oauthlib==3.2.2 ; python_version >= "3.8" and python_version < "3.11"
 opencv-python==4.7.0.68 ; python_version >= "3.8" and python_version < "3.11"
+opt-einsum==3.3.0 ; python_version >= "3.8" and python_version < "3.11"
+packaging==22.0 ; python_version >= "3.8" and python_version < "3.11"
 pathtools==0.1.2 ; python_version >= "3.8" and python_version < "3.11"
-pillow==9.3.0 ; python_version >= "3.8" and python_version < "3.11"
+pillow==9.4.0 ; python_version >= "3.8" and python_version < "3.11"
 proglog==0.1.10 ; python_version >= "3.8" and python_version < "3.11"
 promise==2.3 ; python_version >= "3.8" and python_version < "3.11"
-protobuf==3.20.3 ; python_version < "3.11" and python_version >= "3.8"
+protobuf==3.19.6 ; python_version < "3.11" and python_version >= "3.8"
 psutil==5.9.4 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1-modules==0.2.8 ; python_version >= "3.8" and python_version < "3.11"
 pyasn1==0.4.8 ; python_version >= "3.8" and python_version < "3.11"
@@ -71,6 +80,9 @@ smmap==5.0.0 ; python_version >= "3.8" and python_version < "3.11"
 tensorboard-data-server==0.6.1 ; python_version >= "3.8" and python_version < "3.11"
 tensorboard-plugin-wit==1.8.1 ; python_version >= "3.8" and python_version < "3.11"
 tensorboard==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-estimator==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
+tensorflow-io-gcs-filesystem==0.29.0 ; python_version >= "3.8" and python_version < "3.11" and platform_machine != "arm64" or python_version >= "3.8" and python_version < "3.11" and platform_system != "Darwin"
+tensorflow==2.11.0 ; python_version >= "3.8" and python_version < "3.11"
 termcolor==2.1.1 ; python_version >= "3.8" and python_version < "3.11"
 torch==1.13.1 ; python_version >= "3.8" and python_version < "3.11"
 tqdm==4.64.1 ; python_version >= "3.8" and python_version < "3.11"
@@ -79,4 +91,5 @@ urllib3==1.26.13 ; python_version >= "3.8" and python_version < "3.11"
 wandb==0.13.7 ; python_version >= "3.8" and python_version < "3.11"
 werkzeug==2.2.2 ; python_version >= "3.8" and python_version < "3.11"
 wheel==0.38.4 ; python_version >= "3.8" and python_version < "3.11"
+wrapt==1.14.1 ; python_version >= "3.8" and python_version < "3.11"
 zipp==3.11.0 ; python_version >= "3.8" and python_version < "3.10"


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 73820fd..6157a01 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -11,7 +11,34 @@ on:
       - '.pre-commit-config.yaml'
       - 'CITATION.cff'
 jobs:
-  tests:
+
+  tests-codes-consistency:
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10"]
+        os: [ubuntu-22.04]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v3
+      - name: Setup Python # Set Python version
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install pip dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements/requirements-dev.txt
+      - name: Test with pytest
+        run: pytest tests/test_codes_consistency.py --doctest-modules --junitxml=junit/tests-codes-consistency-results-${{ matrix.python-version }}.xml
+      - name: Upload pytest test results
+        uses: actions/upload-artifact@v3
+        with:
+          name: pytest-tests-codes-consistency-results-${{ matrix.python-version }}
+          path: junit/tests-codes-consistency-results-${{ matrix.python-version }}.xml
+        if: ${{ always() }}
+
+  tests-torch:
     strategy:
       fail-fast: false
       matrix:
@@ -24,10 +51,10 @@ jobs:
         uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
-      # Install pip and pytest
       - name: Install linux dependencies
         if: runner.os == 'Linux'
         run: |
+          sudo apt-get update && \
           sudo apt-get -y install wget unzip \
             git software-properties-common \
             libgl1-mesa-dev \
@@ -41,13 +68,52 @@ jobs:
       - name: Install pip dependencies
         run: |
           python -m pip install --upgrade pip
-          pip install -r requirements/requirements-dev.txt
+          pip install -r requirements/requirements-torch-dev.txt
+      - name: Test with pytest
+        run: pytest tests/test_*_torch.py --doctest-modules --junitxml=junit/tests-torch-results-${{ matrix.python-version }}.xml
+      - name: Upload pytest test results
+        uses: actions/upload-artifact@v3
+        with:
+          name: pytest-tests-torch-results-${{ matrix.python-version }}
+          path: junit/tests-torch-results-${{ matrix.python-version }}.xml
+        if: ${{ always() }}
+
+  tests-tf:
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.8", "3.9", "3.10"]
+        os: [ubuntu-22.04, macos-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v3
+      - name: Setup Python # Set Python version
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install linux dependencies
+        if: runner.os == 'Linux'
+        run: |
+          sudo apt-get update && \
+          sudo apt-get -y install wget unzip \
+            git software-properties-common \
+            libgl1-mesa-dev \
+            libgl1-mesa-glx \
+            libglew-dev \
+            libosmesa6-dev patchelf swig
+      - name: Install macOS dependencies
+        if: runner.os == 'macOS'
+        run: |
+          brew install gcc@9
+      - name: Install pip dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements/requirements-tf-dev.txt
       - name: Test with pytest
-        run: pytest --doctest-modules --junitxml=junit/test-results-${{ matrix.python-version }}.xml
+        run: pytest tests/test_*_tf.py --doctest-modules --junitxml=junit/tests-tf-results-${{ matrix.python-version }}.xml
       - name: Upload pytest test results
         uses: actions/upload-artifact@v3
         with:
-          name: pytest-results-${{ matrix.python-version }}
-          path: junit/test-results-${{ matrix.python-version }}.xml
-        # Use always() to always run this step to publish test results when there are test failures
+          name: pytest-tests-tf-results-${{ matrix.python-version }}
+          path: junit/tests-tf-results-${{ matrix.python-version }}.xml
         if: ${{ always() }}
diff --git a/tests/test_atari.py b/tests/test_atari_tf.py
similarity index 78%
rename from tests/test_atari.py
rename to tests/test_atari_tf.py
index e90ab02..484ca01 100644
--- a/tests/test_atari.py
+++ b/tests/test_atari_tf.py
@@ -3,11 +3,10 @@
 import subprocess
 
 
-def test_dqn_atari() -> None:
+def test_dqn_atari_tf() -> None:
     subprocess.run(
-        "python abcdrl/dqn_atari.py"
+        "python abcdrl/dqn_atari_tf.py"
         + " --env-id BreakoutNoFrameskip-v4"
-        + " --device auto"
         + " --num-envs 2"
         + " --learning-starts 8"
         + " --total-timesteps 32"
diff --git a/tests/test_atari_torch.py b/tests/test_atari_torch.py
new file mode 100644
index 0000000..f0ab85e
--- /dev/null
+++ b/tests/test_atari_torch.py
@@ -0,0 +1,18 @@
+from __future__ import annotations
+
+import subprocess
+
+
+def test_dqn_atari_torch() -> None:
+    subprocess.run(
+        "python abcdrl/dqn_atari_torch.py"
+        + " --env-id BreakoutNoFrameskip-v4"
+        + " --num-envs 2"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 10"
+        + " --batch-size 4",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
diff --git a/tests/test_classic_control.py b/tests/test_classic_control_tf.py
similarity index 77%
rename from tests/test_classic_control.py
rename to tests/test_classic_control_tf.py
index efd2451..660a59f 100644
--- a/tests/test_classic_control.py
+++ b/tests/test_classic_control_tf.py
@@ -3,11 +3,10 @@
 import subprocess
 
 
-def test_dqn() -> None:
+def test_dqn_tf() -> None:
     subprocess.run(
-        "python abcdrl/dqn.py"
+        "python abcdrl/dqn_tf.py"
         + " --env-id CartPole-v1"
-        + " --device auto"
         + " --num-envs 2"
         + " --learning-starts 8"
         + " --total-timesteps 32"
@@ -19,11 +18,10 @@ def test_dqn() -> None:
     )
 
 
-def test_ddqn() -> None:
+def test_ddqn_tf() -> None:
     subprocess.run(
-        "python abcdrl/ddqn.py"
+        "python abcdrl/ddqn_tf.py"
         + " --env-id CartPole-v1"
-        + " --device auto"
         + " --num-envs 2"
         + " --learning-starts 8"
         + " --total-timesteps 32"
@@ -35,11 +33,10 @@ def test_ddqn() -> None:
     )
 
 
-def test_pdqn() -> None:
+def test_pdqn_tf() -> None:
     subprocess.run(
-        "python abcdrl/pdqn.py"
+        "python abcdrl/pdqn_tf.py"
         + " --env-id CartPole-v1"
-        + " --device auto"
         + " --num-envs 2"
         + " --learning-starts 8"
         + " --total-timesteps 32"
diff --git a/tests/test_classic_control_torch.py b/tests/test_classic_control_torch.py
new file mode 100644
index 0000000..c06a3f3
--- /dev/null
+++ b/tests/test_classic_control_torch.py
@@ -0,0 +1,48 @@
+from __future__ import annotations
+
+import subprocess
+
+
+def test_dqn_torch() -> None:
+    subprocess.run(
+        "python abcdrl/dqn_torch.py"
+        + " --env-id CartPole-v1"
+        + " --num-envs 2"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 10"
+        + " --batch-size 4",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
+
+
+def test_ddqn_torch() -> None:
+    subprocess.run(
+        "python abcdrl/ddqn_torch.py"
+        + " --env-id CartPole-v1"
+        + " --num-envs 2"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 10"
+        + " --batch-size 4",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
+
+
+def test_pdqn_torch() -> None:
+    subprocess.run(
+        "python abcdrl/pdqn_torch.py"
+        + " --env-id CartPole-v1"
+        + " --num-envs 2"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 10"
+        + " --batch-size 4",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
diff --git a/tests/test_codes_consistency.py b/tests/test_codes_consistency.py
index 8993fe7..14b3fbc 100644
--- a/tests/test_codes_consistency.py
+++ b/tests/test_codes_consistency.py
@@ -2,133 +2,211 @@
 
 import inspect
 
-import abcdrl
-import abcdrl_copy_from
+from abcdrl import (
+    ddpg_torch,
+    ddqn_tf,
+    ddqn_torch,
+    dqn_atari_tf,
+    dqn_atari_torch,
+    dqn_tf,
+    dqn_torch,
+    pdqn_tf,
+    pdqn_torch,
+    ppo_torch,
+    sac_torch,
+    td3_torch,
+)
+from abcdrl.utils import (
+    dqn_all_wrappers_torch,
+    wrapper_eval_step,
+    wrapper_logger_tf,
+    wrapper_logger_torch,
+    wrapper_print_filter,
+    wrapper_save_model,
+)
 
 
 def test_codes_buffer() -> None:
     # ReplayBuffer
-    assert inspect.getsource(abcdrl.dqn.ReplayBuffer) == inspect.getsource(abcdrl.ddqn.ReplayBuffer)
-    assert inspect.getsource(abcdrl.dqn.ReplayBuffer) == inspect.getsource(abcdrl.ddpg.ReplayBuffer)
-    assert inspect.getsource(abcdrl.dqn.ReplayBuffer) == inspect.getsource(abcdrl.td3.ReplayBuffer)
-    assert inspect.getsource(abcdrl.dqn.ReplayBuffer) == inspect.getsource(abcdrl.sac.ReplayBuffer)
+    assert inspect.getsource(dqn_torch.ReplayBuffer) == inspect.getsource(ddqn_torch.ReplayBuffer)
+    assert inspect.getsource(dqn_torch.ReplayBuffer) == inspect.getsource(ddpg_torch.ReplayBuffer)
+    assert inspect.getsource(dqn_torch.ReplayBuffer) == inspect.getsource(td3_torch.ReplayBuffer)
+    assert inspect.getsource(dqn_torch.ReplayBuffer) == inspect.getsource(sac_torch.ReplayBuffer)
+    assert inspect.getsource(dqn_torch.ReplayBuffer) == inspect.getsource(dqn_tf.ReplayBuffer)
+    assert inspect.getsource(dqn_torch.ReplayBuffer) == inspect.getsource(ddqn_tf.ReplayBuffer)
+
+    # ReplayBuffer optimize_memory_usage
+    assert inspect.getsource(dqn_atari_torch.ReplayBuffer) == inspect.getsource(dqn_atari_tf.ReplayBuffer)
+
+    # PrioritizedReplayBuffer
+    assert inspect.getsource(pdqn_torch.PrioritizedReplayBuffer) == inspect.getsource(pdqn_tf.PrioritizedReplayBuffer)
 
 
 def test_codes_network() -> None:
-    assert inspect.getsource(abcdrl.dqn.Network) == inspect.getsource(abcdrl.ddqn.Network)
-    assert inspect.getsource(abcdrl.dqn.Network) == inspect.getsource(abcdrl.pdqn.Network)
+    # torch
+    assert inspect.getsource(dqn_torch.Network) == inspect.getsource(ddqn_torch.Network)
+    assert inspect.getsource(dqn_torch.Network) == inspect.getsource(pdqn_torch.Network)
 
-    assert inspect.getsource(abcdrl.td3.CriticNetwork) == inspect.getsource(abcdrl.sac.CriticNetwork)
+    assert inspect.getsource(td3_torch.CriticNetwork) == inspect.getsource(sac_torch.CriticNetwork)
+
+    # tensorflow
+    assert inspect.getsource(dqn_tf.Network) == inspect.getsource(ddqn_tf.Network)
+    assert inspect.getsource(dqn_tf.Network) == inspect.getsource(pdqn_tf.Network)
 
 
 def test_codes_model() -> None:
-    assert inspect.getsource(abcdrl.dqn.Model) == inspect.getsource(abcdrl.ddqn.Model)
-    assert inspect.getsource(abcdrl.dqn.Model) == inspect.getsource(abcdrl.pdqn.Model)
+    # torch
+    assert inspect.getsource(dqn_torch.Model) == inspect.getsource(ddqn_torch.Model)
+    assert inspect.getsource(dqn_torch.Model) == inspect.getsource(pdqn_torch.Model)
+
+    # tensorflow
+    assert inspect.getsource(dqn_tf.Model) == inspect.getsource(ddqn_tf.Model)
+    assert inspect.getsource(dqn_tf.Model) == inspect.getsource(pdqn_tf.Model)
 
 
 def test_codes_agent() -> None:
-    assert inspect.getsource(abcdrl.dqn.Agent) == inspect.getsource(abcdrl.ddqn.Agent)
-    assert inspect.getsource(abcdrl.ddpg.Agent) == inspect.getsource(abcdrl.td3.Agent)
-    assert inspect.getsource(abcdrl.dqn.Agent) == inspect.getsource(abcdrl.dqn_atari.Agent)
+    # torch
+    assert inspect.getsource(dqn_torch.Agent) == inspect.getsource(ddqn_torch.Agent)
+    assert inspect.getsource(ddpg_torch.Agent) == inspect.getsource(td3_torch.Agent)
+    assert inspect.getsource(dqn_torch.Agent) == inspect.getsource(dqn_atari_torch.Agent)
+
+    # tensorflow
+    assert inspect.getsource(dqn_tf.Agent) == inspect.getsource(ddqn_tf.Agent)
 
 
 def test_codes_trainer() -> None:
-    assert inspect.getsource(abcdrl.dqn.Trainer.__call__) == inspect.getsource(abcdrl.ddqn.Trainer.__call__)
-    assert inspect.getsource(abcdrl.dqn.Trainer.__call__) == inspect.getsource(abcdrl.pdqn.Trainer.__call__)
-    assert inspect.getsource(abcdrl.dqn.Trainer.__call__) == inspect.getsource(abcdrl.ddpg.Trainer.__call__)
-    assert inspect.getsource(abcdrl.dqn.Trainer.__call__) == inspect.getsource(abcdrl.td3.Trainer.__call__)
-    assert inspect.getsource(abcdrl.dqn.Trainer.__call__) == inspect.getsource(abcdrl.sac.Trainer.__call__)
-    assert inspect.getsource(abcdrl.dqn.Trainer.__call__) == inspect.getsource(abcdrl.dqn_atari.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(ddqn_torch.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(pdqn_torch.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(ddpg_torch.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(td3_torch.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(sac_torch.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(dqn_atari_torch.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(ddqn_tf.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(pdqn_tf.Trainer.__call__)
+    assert inspect.getsource(dqn_torch.Trainer.__call__) == inspect.getsource(dqn_atari_tf.Trainer.__call__)
+
+    assert inspect.getsource(dqn_torch.Trainer._run_collect) == inspect.getsource(ddqn_torch.Trainer._run_collect)
+    assert inspect.getsource(dqn_torch.Trainer._run_collect) == inspect.getsource(pdqn_torch.Trainer._run_collect)
+    assert inspect.getsource(dqn_torch.Trainer._run_collect) == inspect.getsource(ddpg_torch.Trainer._run_collect)
+    assert inspect.getsource(dqn_torch.Trainer._run_collect) == inspect.getsource(td3_torch.Trainer._run_collect)
+    assert inspect.getsource(dqn_torch.Trainer._run_collect) == inspect.getsource(sac_torch.Trainer._run_collect)
+    assert inspect.getsource(dqn_torch.Trainer._run_collect) == inspect.getsource(ddqn_tf.Trainer._run_collect)
+    assert inspect.getsource(dqn_torch.Trainer._run_collect) == inspect.getsource(pdqn_tf.Trainer._run_collect)
+    assert inspect.getsource(dqn_atari_torch.Trainer._run_collect) == inspect.getsource(
+        dqn_atari_tf.Trainer._run_collect
+    )
 
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_collect) == inspect.getsource(abcdrl.ddqn.Trainer._run_collect)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_collect) == inspect.getsource(abcdrl.pdqn.Trainer._run_collect)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_collect) == inspect.getsource(abcdrl.ddpg.Trainer._run_collect)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_collect) == inspect.getsource(abcdrl.td3.Trainer._run_collect)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_collect) == inspect.getsource(abcdrl.sac.Trainer._run_collect)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(ddqn_torch.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(ddpg_torch.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(td3_torch.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(sac_torch.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(dqn_atari_torch.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(dqn_tf.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(ddqn_tf.Trainer._run_train)
+    assert inspect.getsource(dqn_atari_torch.Trainer._run_train) == inspect.getsource(dqn_atari_tf.Trainer._run_train)
 
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_train) == inspect.getsource(abcdrl.ddqn.Trainer._run_train)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_train) == inspect.getsource(abcdrl.ddpg.Trainer._run_train)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_train) == inspect.getsource(abcdrl.td3.Trainer._run_train)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_train) == inspect.getsource(abcdrl.sac.Trainer._run_train)
-    assert inspect.getsource(abcdrl.dqn.Trainer._run_train) == inspect.getsource(abcdrl.dqn_atari.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._make_env) == inspect.getsource(ddqn_torch.Trainer._make_env)
+    assert inspect.getsource(dqn_torch.Trainer._make_env) == inspect.getsource(pdqn_torch.Trainer._make_env)
+    assert inspect.getsource(ddpg_torch.Trainer._make_env) == inspect.getsource(td3_torch.Trainer._make_env)
+    assert inspect.getsource(ddpg_torch.Trainer._make_env) == inspect.getsource(sac_torch.Trainer._make_env)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(dqn_tf.Trainer._run_train)
+    assert inspect.getsource(dqn_torch.Trainer._run_train) == inspect.getsource(ddqn_tf.Trainer._run_train)
+    assert inspect.getsource(dqn_atari_torch.Trainer._run_train) == inspect.getsource(dqn_atari_tf.Trainer._run_train)
 
-    assert inspect.getsource(abcdrl.dqn.Trainer._make_env) == inspect.getsource(abcdrl.ddqn.Trainer._make_env)
-    assert inspect.getsource(abcdrl.dqn.Trainer._make_env) == inspect.getsource(abcdrl.pdqn.Trainer._make_env)
-    assert inspect.getsource(abcdrl.ddpg.Trainer._make_env) == inspect.getsource(abcdrl.td3.Trainer._make_env)
-    assert inspect.getsource(abcdrl.ddpg.Trainer._make_env) == inspect.getsource(abcdrl.sac.Trainer._make_env)
+    assert inspect.getsource(dqn_torch.Trainer) == inspect.getsource(ddqn_torch.Trainer)
 
-    assert inspect.getsource(abcdrl.dqn.Trainer) == inspect.getsource(abcdrl.ddqn.Trainer)
+    assert inspect.getsource(dqn_tf.Trainer) == inspect.getsource(ddqn_tf.Trainer)
 
 
 def test_codes_wrapper() -> None:
-    # wrapper_logger
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.dqn.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.ddqn.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.pdqn.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.ddpg.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.td3.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.sac.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.ppo.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(abcdrl.dqn_atari.wrapper_logger)
-    assert inspect.getsource(abcdrl_copy_from.wrapper_logger) == inspect.getsource(
-        abcdrl_copy_from.dqn_all_wrappers.wrapper_logger
+    # wrapper_logger_torch
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        dqn_torch.wrapper_logger_torch
+    )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        ddqn_torch.wrapper_logger_torch
+    )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        pdqn_torch.wrapper_logger_torch
+    )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        ddpg_torch.wrapper_logger_torch
+    )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        td3_torch.wrapper_logger_torch
+    )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        sac_torch.wrapper_logger_torch
+    )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        ppo_torch.wrapper_logger_torch
+    )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        dqn_atari_torch.wrapper_logger_torch
     )
+    assert inspect.getsource(wrapper_logger_torch.wrapper_logger_torch) == inspect.getsource(
+        dqn_all_wrappers_torch.wrapper_logger_torch
+    )
+
+    # wrapper_logger_tf
+    assert inspect.getsource(wrapper_logger_tf.wrapper_logger_tf) == inspect.getsource(dqn_tf.wrapper_logger_tf)
+    assert inspect.getsource(wrapper_logger_tf.wrapper_logger_tf) == inspect.getsource(ddqn_tf.wrapper_logger_tf)
+    assert inspect.getsource(wrapper_logger_tf.wrapper_logger_tf) == inspect.getsource(pdqn_tf.wrapper_logger_tf)
+    assert inspect.getsource(wrapper_logger_tf.wrapper_logger_tf) == inspect.getsource(dqn_atari_tf.wrapper_logger_tf)
 
     # wrapper_filter
-    assert inspect.getsource(abcdrl_copy_from.wrapper_print_filter) == inspect.getsource(
-        abcdrl_copy_from.dqn_all_wrappers.wrapper_print_filter
+    assert inspect.getsource(wrapper_print_filter.wrapper_print_filter) == inspect.getsource(
+        dqn_all_wrappers_torch.wrapper_print_filter
     )
 
     # wrapper_eval_step
-    assert inspect.getsource(abcdrl_copy_from.wrapper_eval_step) == inspect.getsource(
-        abcdrl_copy_from.dqn_all_wrappers.wrapper_eval_step
+    assert inspect.getsource(wrapper_eval_step.wrapper_eval_step) == inspect.getsource(
+        dqn_all_wrappers_torch.wrapper_eval_step
     )
 
     # wrapper_save_model
-    assert inspect.getsource(abcdrl_copy_from.wrapper_save_model) == inspect.getsource(
-        abcdrl_copy_from.dqn_all_wrappers.wrapper_save_model
+    assert inspect.getsource(wrapper_save_model.wrapper_save_model) == inspect.getsource(
+        dqn_all_wrappers_torch.wrapper_save_model
     )
 
 
 def test_codes_example() -> None:
-    assert inspect.getsource(abcdrl.dqn.Trainer) == inspect.getsource(abcdrl_copy_from.dqn_all_wrappers.Trainer)
-    assert inspect.getsource(abcdrl.dqn.Agent) == inspect.getsource(abcdrl_copy_from.dqn_all_wrappers.Agent)
-    assert inspect.getsource(abcdrl.dqn.Algorithm) == inspect.getsource(abcdrl_copy_from.dqn_all_wrappers.Algorithm)
-    assert inspect.getsource(abcdrl.dqn.Model) == inspect.getsource(abcdrl_copy_from.dqn_all_wrappers.Model)
-    assert inspect.getsource(abcdrl.dqn.ReplayBuffer) == inspect.getsource(
-        abcdrl_copy_from.dqn_all_wrappers.ReplayBuffer
-    )
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(
-        abcdrl_copy_from.dqn_all_wrappers.get_space_shape
-    )
+    assert inspect.getsource(dqn_torch.Trainer) == inspect.getsource(dqn_all_wrappers_torch.Trainer)
+    assert inspect.getsource(dqn_torch.Agent) == inspect.getsource(dqn_all_wrappers_torch.Agent)
+    assert inspect.getsource(dqn_torch.Algorithm) == inspect.getsource(dqn_all_wrappers_torch.Algorithm)
+    assert inspect.getsource(dqn_torch.Model) == inspect.getsource(dqn_all_wrappers_torch.Model)
+    assert inspect.getsource(dqn_torch.ReplayBuffer) == inspect.getsource(dqn_all_wrappers_torch.ReplayBuffer)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(dqn_all_wrappers_torch.get_space_shape)
 
 
 def test_codes_other() -> None:
     # get_space_shape()
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(abcdrl.ddqn.get_space_shape)
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(abcdrl.pdqn.get_space_shape)
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(abcdrl.ddpg.get_space_shape)
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(abcdrl.td3.get_space_shape)
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(abcdrl.sac.get_space_shape)
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(abcdrl.ppo.get_space_shape)
-    assert inspect.getsource(abcdrl.dqn.get_space_shape) == inspect.getsource(abcdrl.dqn_atari.get_space_shape)
-
-    # if __name__ == "__main__":
-    dqn_codes = inspect.getsource(abcdrl.dqn)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(ddqn_torch.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(pdqn_torch.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(ddpg_torch.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(td3_torch.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(sac_torch.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(ppo_torch.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(dqn_atari_torch.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(dqn_tf.get_space_shape)
+    assert inspect.getsource(dqn_torch.get_space_shape) == inspect.getsource(dqn_atari_tf.get_space_shape)
+
+    # torch if __name__ == "__main__":
+    dqn_codes = inspect.getsource(dqn_torch)
     dqn_codes = dqn_codes[dqn_codes.find('if __name__ == "__main__":') :]
-    ddqn_codes = inspect.getsource(abcdrl.ddqn)
+    ddqn_codes = inspect.getsource(ddqn_torch)
     ddqn_codes = ddqn_codes[ddqn_codes.find('if __name__ == "__main__":') :]
-    pdqn_codes = inspect.getsource(abcdrl.pdqn)
+    pdqn_codes = inspect.getsource(pdqn_torch)
     pdqn_codes = pdqn_codes[pdqn_codes.find('if __name__ == "__main__":') :]
-    ddpg_codes = inspect.getsource(abcdrl.ddpg)
+    ddpg_codes = inspect.getsource(ddpg_torch)
     ddpg_codes = ddpg_codes[ddpg_codes.find('if __name__ == "__main__":') :]
-    td3_codes = inspect.getsource(abcdrl.td3)
+    td3_codes = inspect.getsource(td3_torch)
     td3_codes = td3_codes[td3_codes.find('if __name__ == "__main__":') :]
-    sac_codes = inspect.getsource(abcdrl.sac)
+    sac_codes = inspect.getsource(sac_torch)
     sac_codes = sac_codes[sac_codes.find('if __name__ == "__main__":') :]
-    ppo_codes = inspect.getsource(abcdrl.ppo)
+    ppo_codes = inspect.getsource(ppo_torch)
     ppo_codes = ppo_codes[ppo_codes.find('if __name__ == "__main__":') :]
-    dqn_atari_codes = inspect.getsource(abcdrl.dqn_atari)
+    dqn_atari_codes = inspect.getsource(dqn_atari_torch)
     dqn_atari_codes = dqn_atari_codes[dqn_atari_codes.find('if __name__ == "__main__":') :]
     assert dqn_codes == ddqn_codes
     assert dqn_codes == pdqn_codes
@@ -137,3 +215,16 @@ def test_codes_other() -> None:
     assert dqn_codes == sac_codes
     assert dqn_codes == ppo_codes
     assert dqn_codes == dqn_atari_codes
+
+    # tensorflow if __name__ == "__main__":
+    dqn_codes = inspect.getsource(dqn_tf)
+    dqn_codes = dqn_codes[dqn_codes.find('if __name__ == "__main__":') :]
+    ddqn_codes = inspect.getsource(ddqn_tf)
+    ddqn_codes = ddqn_codes[ddqn_codes.find('if __name__ == "__main__":') :]
+    pdqn_codes = inspect.getsource(pdqn_tf)
+    pdqn_codes = pdqn_codes[pdqn_codes.find('if __name__ == "__main__":') :]
+    dqn_atari_codes = inspect.getsource(dqn_atari_tf)
+    dqn_atari_codes = dqn_atari_codes[dqn_atari_codes.find('if __name__ == "__main__":') :]
+    assert dqn_codes == ddqn_codes
+    assert dqn_codes == pdqn_codes
+    assert dqn_codes == dqn_atari_codes
diff --git a/tests/test_examples_torch.py b/tests/test_examples_torch.py
new file mode 100644
index 0000000..c376545
--- /dev/null
+++ b/tests/test_examples_torch.py
@@ -0,0 +1,45 @@
+from __future__ import annotations
+
+import subprocess
+
+
+def test_example_dqn_all_wrapper_torch() -> None:
+    subprocess.run(
+        "python abcdrl/utils/dqn_all_wrappers_torch.py"
+        + " --env-id CartPole-v1"
+        + " --num-envs 2"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 10"
+        + " --batch-size 4"
+        + " --eval-frequency 5"
+        + " --num-steps-eval 1"
+        + " --save-frequency 16",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
+
+
+def test_example_eval_model_torch() -> None:
+    subprocess.run(
+        "python abcdrl/utils/dqn_all_wrappers_torch.py"
+        + " --exp_name test_eval_dqn"
+        + " --env-id CartPole-v1"
+        + " --num-envs 1"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 64"
+        + " --batch-size 4"
+        + " --save-frequency 16",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
+
+    subprocess.run(
+        "python example_eval_model.py" + " --model-path models/test_eval_dqn/s16.agent" + " --total_timesteps 100",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
diff --git a/tests/test_mujoco.py b/tests/test_mujoco_torch.py
similarity index 77%
rename from tests/test_mujoco.py
rename to tests/test_mujoco_torch.py
index a0c629d..e43e05c 100644
--- a/tests/test_mujoco.py
+++ b/tests/test_mujoco_torch.py
@@ -3,11 +3,10 @@
 import subprocess
 
 
-def test_ddpg() -> None:
+def test_ddpg_torch() -> None:
     subprocess.run(
-        "python abcdrl/ddpg.py"
+        "python abcdrl/ddpg_torch.py"
         + " --env-id Hopper-v4"
-        + " --device auto"
         + " --num-envs 2"
         + " --learning-starts 64"
         + " --total-timesteps 256"
@@ -19,11 +18,10 @@ def test_ddpg() -> None:
     )
 
 
-def test_td3() -> None:
+def test_td3_torch() -> None:
     subprocess.run(
-        "python abcdrl/td3.py"
+        "python abcdrl/td3_torch.py"
         + " --env-id Hopper-v4"
-        + " --device auto"
         + " --num-envs 2"
         + " --learning-starts 64"
         + " --total-timesteps 256"
@@ -35,11 +33,10 @@ def test_td3() -> None:
     )
 
 
-def test_sac() -> None:
+def test_sac_torch() -> None:
     subprocess.run(
-        "python abcdrl/sac.py"
+        "python abcdrl/sac_torch.py"
         + " --env-id Hopper-v4"
-        + " --device auto"
         + " --num-envs 2"
         + " --learning-starts 64"
         + " --total-timesteps 256"
@@ -51,11 +48,10 @@ def test_sac() -> None:
     )
 
 
-def test_ppo() -> None:
+def test_ppo_torch() -> None:
     subprocess.run(
-        "python abcdrl/ppo.py"
+        "python abcdrl/ppo_torch.py"
         + " --env-id Hopper-v4"
-        + " --device auto"
         + " --num-envs 2"
         + " --num-steps 64"
         + " --num-minibatches 16"
diff --git a/tests/test_utils.py b/tests/test_utils.py
deleted file mode 100644
index 6d97a7b..0000000
--- a/tests/test_utils.py
+++ /dev/null
@@ -1,103 +0,0 @@
-from __future__ import annotations
-
-import subprocess
-
-
-def test_benchmark() -> None:
-    try:
-        subprocess.run(
-            "python benchmark.py",
-            shell=True,
-            check=True,
-            timeout=3,
-        )
-    except subprocess.TimeoutExpired:
-        pass
-
-
-def test_example_all_wrapper() -> None:
-    subprocess.run(
-        "python abcdrl_copy_from/dqn_all_wrappers.py"
-        + " --env-id CartPole-v1"
-        + " --device auto"
-        + " --num-envs 2"
-        + " --learning-starts 8"
-        + " --total-timesteps 32"
-        + " --buffer-size 10"
-        + " --batch-size 4"
-        + " --eval-frequency 5"
-        + " --num-steps-eval 1"
-        + " --save-frequency 16",
-        shell=True,
-        check=True,
-        timeout=100,
-    )
-
-
-def test_example_eval_model() -> None:
-    subprocess.run(
-        "python abcdrl_copy_from/dqn_all_wrappers.py"
-        + " --exp_name test_eval_dqn"
-        + " --env-id CartPole-v1"
-        + " --device auto"
-        + " --num-envs 1"
-        + " --learning-starts 8"
-        + " --total-timesteps 32"
-        + " --buffer-size 64"
-        + " --batch-size 4"
-        + " --save-frequency 16",
-        shell=True,
-        check=True,
-        timeout=100,
-    )
-
-    subprocess.run(
-        "python example_eval_model.py" + " --model-path models/test_eval_dqn/s16.agent" + " --total_timesteps 100",
-        shell=True,
-        check=True,
-        timeout=100,
-    )
-
-
-def test_capture_video() -> None:
-    subprocess.run(
-        "python abcdrl/dqn.py"
-        + " --env-id CartPole-v1"
-        + " --device auto"
-        + " --num-envs 2"
-        + " --learning-starts 8"
-        + " --total-timesteps 32"
-        + " --buffer-size 10"
-        + " --batch-size 4"
-        + " --capture-video True",
-        shell=True,
-        check=True,
-        timeout=100,
-    )
-
-
-def test_wandb_track() -> None:
-    online_flag = False
-    re = subprocess.run("wandb status", shell=True, check=True, capture_output=True, timeout=10)
-    if "disabled" not in str(re.stdout):
-        subprocess.run("wandb offline", shell=True, check=True, timeout=10)
-        online_flag = True
-
-    try:
-        subprocess.run(
-            "python abcdrl/dqn.py"
-            + " --env-id CartPole-v1"
-            + " --device auto"
-            + " --num-envs 2"
-            + " --learning-starts 8"
-            + " --total-timesteps 32"
-            + " --buffer-size 10"
-            + " --batch-size 4"
-            + " --capture-video True",
-            shell=True,
-            check=True,
-            timeout=100,
-        )
-    finally:
-        if online_flag:
-            subprocess.run("wandb online", shell=True, check=True, timeout=10)
diff --git a/tests/test_utils_tf.py b/tests/test_utils_tf.py
new file mode 100644
index 0000000..fec3d7d
--- /dev/null
+++ b/tests/test_utils_tf.py
@@ -0,0 +1,57 @@
+from __future__ import annotations
+
+import subprocess
+
+
+def test_benchmark_tf() -> None:
+    try:
+        subprocess.run(
+            "python benchmark.py --algs dqn --env-id CartPole-v1 --frameworks tf",
+            shell=True,
+            check=True,
+            timeout=3,
+        )
+    except subprocess.TimeoutExpired:
+        pass
+
+
+def test_capture_video_tf() -> None:
+    subprocess.run(
+        "python abcdrl/dqn_tf.py"
+        + " --env-id CartPole-v1"
+        + " --num-envs 2"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 10"
+        + " --batch-size 4"
+        + " --capture-video True",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
+
+
+def test_wandb_track_tf() -> None:
+    online_flag = False
+    re = subprocess.run("wandb status", shell=True, check=True, capture_output=True, timeout=10)
+    if "disabled" not in str(re.stdout):
+        subprocess.run("wandb offline", shell=True, check=True, timeout=10)
+        online_flag = True
+
+    try:
+        subprocess.run(
+            "python abcdrl/dqn_tf.py"
+            + " --env-id CartPole-v1"
+            + " --num-envs 2"
+            + " --learning-starts 8"
+            + " --total-timesteps 32"
+            + " --buffer-size 10"
+            + " --batch-size 4"
+            + " --capture-video True",
+            shell=True,
+            check=True,
+            timeout=100,
+        )
+    finally:
+        if online_flag:
+            subprocess.run("wandb online", shell=True, check=True, timeout=10)
diff --git a/tests/test_utils_torch.py b/tests/test_utils_torch.py
new file mode 100644
index 0000000..5000ff4
--- /dev/null
+++ b/tests/test_utils_torch.py
@@ -0,0 +1,57 @@
+from __future__ import annotations
+
+import subprocess
+
+
+def test_benchmark_torch() -> None:
+    try:
+        subprocess.run(
+            "python benchmark.py --algs dqn --env-id CartPole-v1 --frameworks torch",
+            shell=True,
+            check=True,
+            timeout=3,
+        )
+    except subprocess.TimeoutExpired:
+        pass
+
+
+def test_capture_video_torch() -> None:
+    subprocess.run(
+        "python abcdrl/dqn_torch.py"
+        + " --env-id CartPole-v1"
+        + " --num-envs 2"
+        + " --learning-starts 8"
+        + " --total-timesteps 32"
+        + " --buffer-size 10"
+        + " --batch-size 4"
+        + " --capture-video True",
+        shell=True,
+        check=True,
+        timeout=100,
+    )
+
+
+def test_wandb_track_torch() -> None:
+    online_flag = False
+    re = subprocess.run("wandb status", shell=True, check=True, capture_output=True, timeout=10)
+    if "disabled" not in str(re.stdout):
+        subprocess.run("wandb offline", shell=True, check=True, timeout=10)
+        online_flag = True
+
+    try:
+        subprocess.run(
+            "python abcdrl/dqn_torch.py"
+            + " --env-id CartPole-v1"
+            + " --num-envs 2"
+            + " --learning-starts 8"
+            + " --total-timesteps 32"
+            + " --buffer-size 10"
+            + " --batch-size 4"
+            + " --capture-video True",
+            shell=True,
+            check=True,
+            timeout=100,
+        )
+    finally:
+        if online_flag:
+            subprocess.run("wandb online", shell=True, check=True, timeout=10)
diff --git a/tests/test_wrappers_tf.py b/tests/test_wrappers_tf.py
new file mode 100644
index 0000000..6d0e3e3
--- /dev/null
+++ b/tests/test_wrappers_tf.py
@@ -0,0 +1,66 @@
+from __future__ import annotations
+
+from typing import Any, Callable, Generator
+
+from abcdrl import ddqn_tf, dqn_tf, pdqn_tf
+from abcdrl.utils import (
+    wrapper_eval_step,
+    wrapper_logger_tf,
+    wrapper_print_filter,
+    wrapper_save_model,
+)
+
+
+def set_all_wrappers(
+    func: Callable[..., Generator[dict[str, Any], None, None]]
+) -> Callable[..., Generator[dict[str, Any], None, None]]:
+    func = wrapper_eval_step.wrapper_eval_step(func)  # type: ignore[assignment]
+    func = wrapper_logger_tf.wrapper_logger_tf(func)  # type: ignore[assignment]
+    func = wrapper_save_model.wrapper_save_model(func)  # type: ignore[assignment]
+    func = wrapper_print_filter.wrapper_print_filter(func)  # type: ignore[assignment]
+    return func
+
+
+def test_dqn_tf_wrappers() -> None:
+    Trainer = dqn_tf.Trainer
+    Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
+    trainer = Trainer(
+        env_id="CartPole-v1",
+        num_envs=2,
+        learning_starts=8,
+        total_timesteps=32,
+        buffer_size=10,
+        batch_size=4,
+    )
+    for _ in trainer(eval_frequency=5, num_steps_eval=1, save_frequency=16):  # type: ignore[call-arg]
+        pass
+
+
+def test_ddqn_tf_wrappers() -> None:
+    Trainer = ddqn_tf.Trainer
+    Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
+    trainer = Trainer(
+        env_id="CartPole-v1",
+        num_envs=2,
+        learning_starts=8,
+        total_timesteps=32,
+        buffer_size=10,
+        batch_size=4,
+    )
+    for _ in trainer(eval_frequency=5, num_steps_eval=1, save_frequency=16):  # type: ignore[call-arg]
+        pass
+
+
+def test_pdqn_tf_wrappers() -> None:
+    Trainer = pdqn_tf.Trainer
+    Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
+    trainer = Trainer(
+        env_id="CartPole-v1",
+        num_envs=2,
+        learning_starts=8,
+        total_timesteps=32,
+        buffer_size=10,
+        batch_size=4,
+    )
+    for _ in trainer(eval_frequency=5, num_steps_eval=1, save_frequency=16):  # type: ignore[call-arg]
+        pass
diff --git a/tests/test_copy_from.py b/tests/test_wrappers_torch.py
similarity index 74%
rename from tests/test_copy_from.py
rename to tests/test_wrappers_torch.py
index 47c233f..9aafd42 100644
--- a/tests/test_copy_from.py
+++ b/tests/test_wrappers_torch.py
@@ -2,26 +2,39 @@
 
 from typing import Any, Callable, Generator
 
-import abcdrl
-import abcdrl_copy_from
+from abcdrl import (
+    ddpg_torch,
+    ddqn_torch,
+    dqn_atari_torch,
+    dqn_torch,
+    pdqn_torch,
+    ppo_torch,
+    sac_torch,
+    td3_torch,
+)
+from abcdrl.utils import (
+    wrapper_eval_step,
+    wrapper_logger_torch,
+    wrapper_print_filter,
+    wrapper_save_model,
+)
 
 
 def set_all_wrappers(
     func: Callable[..., Generator[dict[str, Any], None, None]]
 ) -> Callable[..., Generator[dict[str, Any], None, None]]:
-    func = abcdrl_copy_from.wrapper_eval_step(func)  # type: ignore[assignment]
-    func = abcdrl_copy_from.wrapper_logger(func)  # type: ignore[assignment]
-    func = abcdrl_copy_from.wrapper_save_model(func)  # type: ignore[assignment]
-    func = abcdrl_copy_from.wrapper_print_filter(func)  # type: ignore[assignment]
+    func = wrapper_eval_step.wrapper_eval_step(func)  # type: ignore[assignment]
+    func = wrapper_logger_torch.wrapper_logger_torch(func)  # type: ignore[assignment]
+    func = wrapper_save_model.wrapper_save_model(func)  # type: ignore[assignment]
+    func = wrapper_print_filter.wrapper_print_filter(func)  # type: ignore[assignment]
     return func
 
 
-def test_dqn_wrappers() -> None:
-    Trainer = abcdrl.dqn.Trainer
+def test_dqn_torch_wrappers() -> None:
+    Trainer = dqn_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="CartPole-v1",
-        device="auto",
         num_envs=2,
         learning_starts=8,
         total_timesteps=32,
@@ -32,12 +45,11 @@ def test_dqn_wrappers() -> None:
         pass
 
 
-def test_dqn_atari_wrappers() -> None:
-    Trainer = abcdrl.dqn_atari.Trainer
+def test_dqn_atari_torch_wrappers() -> None:
+    Trainer = dqn_atari_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="BreakoutNoFrameskip-v4",
-        device="auto",
         num_envs=2,
         learning_starts=8,
         total_timesteps=32,
@@ -48,12 +60,11 @@ def test_dqn_atari_wrappers() -> None:
         pass
 
 
-def test_ddqn_wrappers() -> None:
-    Trainer = abcdrl.ddqn.Trainer
+def test_ddqn_torch_wrappers() -> None:
+    Trainer = ddqn_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="CartPole-v1",
-        device="auto",
         num_envs=2,
         learning_starts=8,
         total_timesteps=32,
@@ -64,12 +75,11 @@ def test_ddqn_wrappers() -> None:
         pass
 
 
-def test_pdqn_wrappers() -> None:
-    Trainer = abcdrl.pdqn.Trainer
+def test_pdqn_torch_wrappers() -> None:
+    Trainer = pdqn_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="CartPole-v1",
-        device="auto",
         num_envs=2,
         learning_starts=8,
         total_timesteps=32,
@@ -80,12 +90,11 @@ def test_pdqn_wrappers() -> None:
         pass
 
 
-def test_ddpg_wrappers() -> None:
-    Trainer = abcdrl.ddpg.Trainer
+def test_ddpg_torch_wrappers() -> None:
+    Trainer = ddpg_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="Hopper-v4",
-        device="auto",
         num_envs=2,
         learning_starts=64,
         total_timesteps=256,
@@ -96,12 +105,11 @@ def test_ddpg_wrappers() -> None:
         pass
 
 
-def test_td3_wrappers() -> None:
-    Trainer = abcdrl.td3.Trainer
+def test_td3_torch_wrappers() -> None:
+    Trainer = td3_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="Hopper-v4",
-        device="auto",
         num_envs=2,
         learning_starts=64,
         total_timesteps=256,
@@ -112,12 +120,11 @@ def test_td3_wrappers() -> None:
         pass
 
 
-def test_sac_wrappers() -> None:
-    Trainer = abcdrl.sac.Trainer
+def test_sac_torch_wrappers() -> None:
+    Trainer = sac_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="Hopper-v4",
-        device="auto",
         num_envs=2,
         learning_starts=64,
         total_timesteps=256,
@@ -128,12 +135,11 @@ def test_sac_wrappers() -> None:
         pass
 
 
-def test_ppo_wrappers() -> None:
-    Trainer = abcdrl.ppo.Trainer
+def test_ppo_torch_wrappers() -> None:
+    Trainer = ppo_torch.Trainer
     Trainer.__call__ = set_all_wrappers(Trainer.__call__)  # type: ignore[assignment]
     trainer = Trainer(
         env_id="Hopper-v4",
-        device="auto",
         num_envs=2,
         num_steps=64,
         num_minibatches=16,