Repository: thousandbrainsproject/monty_lab
Pull request #34: build: use thousandbrainsproject::tbp.monty as a dependency
Author: login: tristanls
Created: 2025-03-06T18:12:19Z | Merged: 2025-03-17T21:14:23Z

This pull request changes how the conda `monty_lab` environment is created.

Instead of cloning the local `tbp.monty` conda environment and therefore using the local `tbp.monty` code, this change updates `monty_lab` to use the latest published `tbp.monty` version from https://anaconda.org/thousandbrainsproject/tbp.monty.

This was tested locally on my machine via:
```
$ conda env create -f environment.yml --subdir=osx-64
...
$ conda activate monty_lab
...
$ python dmc/run_parallel.py -e dist_agent_1lm
... # ran as expected
```

If you would like to test this on your machine, copy the following into an `environment.yml` file:
```
# environment.yml
name: monty_lab_test
channels:
  - aihabitat
  - pytorch
  - pyg
  - defaults
  - conda-forge
  - thousandbrainsproject

dependencies:
  - thousandbrainsproject::tbp.monty
```
Then:
```
$ conda env create -f environment.yml --subdir=osx-64
# if your are @ramyamounir (linux), then omit the --subdir option
...
$ conda activate monty_lab_test
...
# run what you need to verify things work
```

> [!WARNING]
> This means that you will no longer be able to edit your local `tbp.monty` code and see the effects in `monty_lab`. Instead, if you need to overwrite `tbp.monty` code, you'll need to subclass the desired class and override its method inside `monty_lab`.

> [!NOTE]
> The `import_config_from_monty` used by `randrot_noise_10distinctobj_5lms_dist_agent_profiled` in `experiments/configs/profiled_runs.py` still expects there to be a `~/tbp/tbp.monty` repository from which it is importing a config.

**question**: I am looking at updating how our infrastructure will handle this update. Is it ok for `experiments/configs/profiled_runs.py` to stop working?

**note**: I will wait to merge this pull request until our infrastructure is ready to incorporate this change.


Diff:
diff --git a/README.md b/README.md
index cb5b6ec7..83ab06de 100644
--- a/README.md
+++ b/README.md
@@ -3,8 +3,46 @@
 This is where we (the TBP team) keep our day-to-day experiment files and data analysis scripts. The code in this repository is not unit tested or actively maintained. Some older projects in here are just for informational purposes or to replicate old experiments.
 
 ## Setup
-To run the code in this repository it is best to make a new conda environment by cloning the tbp.monty environment. For setup instructions for this see https://thousandbrainsproject.readme.io/docs/getting-started.
-`conda create --clone tbp.monty -n monty_lab`
+
+To run the code in this repository requires Conda to install the dependencies. For instructions on how to install Conda (Miniconda or Anaconda) on your machine see https://conda.io/projects/conda/en/latest/user-guide/install/index.html.
+
+NOTE: By default, Conda will activate the base environment when you open a new terminal. If you do not want Conda to change your global shell when you open a new terminal, run: `conda config --set auto_activate_base false`.
+
+To setup `monty_lab`, **use the conda commands below**. Make sure to `cd` into the `monty_lab` directory before running these commands.
+
+Note that the commands are slightly different depending on whether you are setting up the environment on an Intel or ARM64 architecture, and whether you are using the zsh shell or another shell.
+
+You can create the environment with the following commands:
+
+### ARM64 (Apple Silicon) (zsh shell)
+```
+conda env create -f environment.yml --subdir=osx-64
+conda init zsh
+conda activate monty_lab
+conda config --env --set subdir osx-64
+```
+
+### ARM64 (Apple Silicon) (bash shell)
+```
+conda env create -f environment.yml --subdir=osx-64
+conda init
+conda activate monty_lab
+conda config --env --set subdir osx-64
+```
+
+### Intel (zsh shell)
+```
+conda env create -f environment.yml
+conda init zsh
+conda activate monty_lab
+```
+
+### Intel (bash shell)
+```
+conda env create -f environment.yml
+conda init
+conda activate monty_lab
+```
 
 Then, you can install any additional dependencies your specific project requires like this:
 
diff --git a/environment.yml b/environment.yml
new file mode 100644
index 00000000..13bb95a0
--- /dev/null
+++ b/environment.yml
@@ -0,0 +1,28 @@
+# This file may be used to create an environment using conda:
+#
+# ## Miniconda or Anaconda
+#
+# Linux
+#     $ conda env create -f environment.yml
+# MacOS
+#     $ conda env create -f environment.yml --subdir=osx-64
+# If you are using the zsh shell, run:
+#     $ conda init zsh
+# Or, if you are using a different shell, run:
+#     $ conda init
+# After init, if you do not want conda to change your global shell when
+# you open a new terminal, run:
+#     $ conda config --set auto_activate_base false
+# Finally, activate the environment with:
+#     $ conda activate monty_lab
+name: monty_lab
+channels:
+  - aihabitat
+  - pytorch
+  - pyg
+  - defaults
+  - conda-forge
+  - thousandbrainsproject
+
+dependencies:
+  - thousandbrainsproject::tbp.monty


Reviews:
user: login: vkakerbeck
body: Nice, this looks good.

Regarding profiled_runs, what is the reason it might not work anymore? Is it because it uses `import_config_from_monty`? Generally we should retain the ability to run profiled experiments from a different repository (since this will not be included in the benchmark configs but is very useful for figuring out performance bottlenecks).
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T08:43:00Z
user: login: nielsleadholm
body: Looks great, thanks for adding this!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T12:14:44Z
user: login: tristanls
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T23:06:24Z
user: login: hlee9212
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-10T17:04:24Z
user: login: ramyamounir
body: Looks good, thanks.
state: APPROVED
author_association: NONE
submitted_at: 2025-03-10T18:46:51Z
user: login: scottcanoe
body: All looks good to me, thanks!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-10T18:53:39Z

Review comments:
diff_hunk: @@ -3,8 +3,46 @@
 This is where we (the TBP team) keep our day-to-day experiment files and data analysis scripts. The code in this repository is not unit tested or actively maintained. Some older projects in here are just for informational purposes or to replicate old experiments.
 
 ## Setup
-To run the code in this repository it is best to make a new conda environment by cloning the tbp.monty environment. For setup instructions for this see https://thousandbrainsproject.readme.io/docs/getting-started.
-`conda create --clone tbp.monty -n monty_lab`
+
+To run the code in this repository requiers Conda to install the dependencies. For instructions on how to install Conda (Miniconda or Anaconda) on your machine see https://conda.io/projects/conda/en/latest/user-guide/install/index.html.
path: README.md
user: login: vkakerbeck
body: ```suggestion
To run the code in this repository requires Conda to install the dependencies. For instructions on how to install Conda (Miniconda or Anaconda) on your machine see https://conda.io/projects/conda/en/latest/user-guide/install/index.html.
```
created_at: 2025-03-07T08:38:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -3,8 +3,46 @@
 This is where we (the TBP team) keep our day-to-day experiment files and data analysis scripts. The code in this repository is not unit tested or actively maintained. Some older projects in here are just for informational purposes or to replicate old experiments.
 
 ## Setup
-To run the code in this repository it is best to make a new conda environment by cloning the tbp.monty environment. For setup instructions for this see https://thousandbrainsproject.readme.io/docs/getting-started.
-`conda create --clone tbp.monty -n monty_lab`
+
+To run the code in this repository requiers Conda to install the dependencies. For instructions on how to install Conda (Miniconda or Anaconda) on your machine see https://conda.io/projects/conda/en/latest/user-guide/install/index.html.
path: README.md
user: login: tristanls
body: Thank you, updated.
created_at: 2025-03-07T23:06:24Z
author_association: CONTRIBUTOR

PR comments:
user: login: codeallthethingz
created_at: 2025-03-06T18:31:53Z
body: I'm going to abstain from this PR as I don't have experience with the labs workflow.
author_association: NONE
user: login: hlee9212
created_at: 2025-03-06T18:33:06Z
body: AFAIK, I think the `monty_lab/experiments` folder is quite an "old" folder. The folders that are currently active in monty_lab are:

1. `benchmark_vit` (has it's own environment so doesn't matter): When it becomes its own repo, it will already have environment.yml.

2. `dmc` (I use `monty_lab` conda environment for this): When it becomes its own repo, it should have environment.yml with versioned tbp.monty for future reproducibility.

4. `floppy` (also use `monty_lab` conda environment for this): When it becomes its own repo, it should have environment.yml with versioned tbp.monty (and ideally it can support newer versions of tbp.monty but depends on maintenance).

author_association: CONTRIBUTOR
user: login: tristanls
created_at: 2025-03-07T23:04:32Z
body: @vkakerbeck, my question regarding keeping profiled runs had to do with whether I should be checking out both the `monty_lab` repo and the `tbp.monty` repo (even though `monty_lab` uses `tbp.monty` as a dependency) in our infrastructure setup. Since you're saying that we should retain the profiled runs capability, I will continue to checkout both repos in our infrastructure setup for `monty_lab` so that profiled runs will continue to work.
author_association: CONTRIBUTOR
user: login: tristanls
created_at: 2025-03-10T23:34:07Z
body: Thank you all for the approvals. As mentioned previously, I will wait to merge this pull request until our infrastructure is ready to incorporate this change.
author_association: CONTRIBUTOR