Repository: getzlab/AnnoMate (language: python)

Issue:
Dev branch
Update to version 1.0.0

- Instead of a direct pickle file, users will pass a directory path. AnnoMate will automatically populate the directory with `data.pkl` and `metadata_config.yaml` file.
- The pickle file will NOT be immediately "frozen" on instantiation with `set_review_data(...)`. Instead, the user will be in TEST MODE until they press the `freeze data` button in the dashboard. The state of the reviewer and a timestamp of when the data is frozen will be stored in the `metadata_config.yaml` file.
- Removed stray print statements
- Updated documentation
- Bumped flask and Werkzeug version, old versions had security issues.


Discussion / hints:

Fix patch:
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000..1b70ec5
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,10 @@
+FROM gcr.io/broad-getzlab-workflows/base_image:v0.0.6 
+# python 3.8
+
+WORKDIR /build
+
+RUN pip install --upgrade pip
+RUN pip install AnnoMate==1.0.0
+
+# Install prebuilt reviewer environments
+RUN git clone https://github.com/getzlab/AnnoMate.git
\ No newline at end of file
diff --git a/README.md b/README.md
index 6e09fcb..7a68210 100644
--- a/README.md
+++ b/README.md
@@ -12,7 +12,7 @@ A package for using and creating interactive dashboards for manual review.
 
 This is _highly_ recommended to manage different dependencies required by different reviewers.
 
-1. Install conda
+1. Install conda if you do not have it already
 
     Credit to Raymond Chu this article: https://medium.com/google-cloud/set-up-anaconda-under-google-cloud-vm-on-windows-f71fc1064bd7
 
@@ -35,7 +35,7 @@ This is _highly_ recommended to manage different dependencies required by differ
     conda create --name <your_env> python==<py_version>
     ```
 
-    `<your_env>` is the name of your environment (ie purity_review_env). Check the corresponding reviewer's `setup.py` file to get the proper python version for `py_version`.
+    `<your_env>` is the name of your environment (ie purity_review_env). Check the corresponding reviewer's `setup.py` file to get the proper python version for `py_version`. Reviewers have been tested on `3.8` and `3.9`. 
 
 3. Add conda environment to ipykernel 
 
@@ -50,7 +50,6 @@ This is _highly_ recommended to manage different dependencies required by differ
 
     When you open a jupyter notebook, you can change the environment the notebook cells are run in to `<your_env>`
 
-
 ### Install AnnoMate with pip
 
 If you are developing a brand new reviewer, you can install from PyPi
@@ -70,13 +69,53 @@ cd AnnoMate
 pip install -e .
 ```
 
+### Install with Conda
+
+Assuming you already have a conda environment:
+```
+conda activate <your_env>
+conda env update --name <your_env> --file annomate_conda_environment.yml
+```
+
+If you have not made a new conda environment:
+```
+conda env create --file annomate_conda_environment.yml --name <your_env>
+```
+Make sure to add conda environment to ipykernel (see **3. Add conda environment to ipykernel**)
+
+### Run with a Docker container
+
+Below are the commands needed to run the tutorial notebooks in a docker container. You need to open at least 2 ports if you are using Mac or Windows:
+1. A port to open jupyter lab (in this case, `<jupyter_port>`)
+2. A port to open the dash app (`<dash_port>`)
+```
+docker run -it -p <jupyter_port>:<jupyter_port> -p <dash_port>:<dash_port> gcr.io/broad-getzlab-workflows/annomate:tests_v381
+cd AnnoMate/tutorial_notebooks
+jupyter lab --ip 0.0.0.0 --port <jupyter_port> --no-browser --allow-root
+```
+In the notebooks, you can set the port to open the Dash app in `reviewer.run(port=<dash_port>, ...)`.
+
+If you are running on Linux, you can open all ports.
+```
+docker run -it --network host gcr.io/broad-getzlab-workflows/annomate:tests_v381
+cd AnnoMate/tutorial_notebooks
+jupyter lab --ip 0.0.0.0 --port <jupyter_port> --no-browser --allow-root
+```
+
 ### Tutorials and Documentation
 
-See a more detailed tutorial in `tutorial_notebooks/Intro_to_Reviewers.ipynb`.
+To run tutorial notebooks, clone the repository if you have not already. If you are running from the docker container, the repo has already been cloned.
+
+```
+git clone git@github.com:getzlab/AnnoMate.git
+cd AnnoMate/tutorial_notebooks
+```
 
-View the catalog of existing reviewers at [catalog/ReviewerCatalog.ipynb](https://github.com/getzlab/AnnoMate/blob/master/catalog/ReviewerCatalog.ipynb).
+Make sure to set the notebook kernels to the environment with the `AnnoMate` package.
 
-For developers, see `tutorial_notebooks/Developer_Jupyter_Reviewer_Tutorial.ipynb`.
+- See a more detailed tutorial in `tutorial_notebooks/Intro_to_Reviewers.ipynb`. 
+- View the catalog of existing reviewers at [catalog/ReviewerCatalog.ipynb](https://github.com/getzlab/AnnoMate/blob/master/catalog/ReviewerCatalog.ipynb).
+- For developers, see `tutorial_notebooks/Developer_Jupyter_Reviewer_Tutorial.ipynb`.
 
 ## Why AnnoMate
 ### Why and how we review data
diff --git a/annomate_conda_environment.yml b/annomate_conda_environment.yml
new file mode 100644
index 0000000..59e495d
--- /dev/null
+++ b/annomate_conda_environment.yml
@@ -0,0 +1,195 @@
+name: clean_annomate_env
+channels:
+  - defaults
+dependencies:
+  - backcall=0.2.0=pyhd3eb1b0_0
+  - ca-certificates=2023.05.30=hecd8cb5_0
+  - decorator=5.1.1=pyhd3eb1b0_0
+  - importlib_metadata=6.0.0=hd3eb1b0_0
+  - jupyter_client=8.1.0=py38hecd8cb5_0
+  - jupyter_core=5.3.0=py38hecd8cb5_0
+  - libcxx=14.0.6=h9765a3e_0
+  - libedit=3.1.20221030=h6c40b1e_0
+  - libffi=3.2.1=h0a44026_1007
+  - libsodium=1.0.18=h1de35cc_0
+  - matplotlib-inline=0.1.6=py38hecd8cb5_0
+  - ncurses=6.4=hcec6c5f_0
+  - nest-asyncio=1.5.6=py38hecd8cb5_0
+  - openssl=1.1.1u=hca72f7f_0
+  - parso=0.8.3=pyhd3eb1b0_0
+  - pexpect=4.8.0=pyhd3eb1b0_3
+  - pickleshare=0.7.5=pyhd3eb1b0_1003
+  - pip=23.1.2=py38hecd8cb5_0
+  - ptyprocess=0.7.0=pyhd3eb1b0_2
+  - pure_eval=0.2.2=pyhd3eb1b0_0
+  - pygments=2.15.1=py38hecd8cb5_1
+  - python=3.8.0=h359304d_2
+  - python-dateutil=2.8.2=pyhd3eb1b0_0
+  - pyzmq=25.1.0=py38hcec6c5f_0
+  - readline=7.0=h1de35cc_5
+  - setuptools=67.8.0=py38hecd8cb5_0
+  - six=1.16.0=pyhd3eb1b0_1
+  - sqlite=3.33.0=hffcf06c_0
+  - stack_data=0.2.0=pyhd3eb1b0_0
+  - tk=8.6.12=h5d9f67b_0
+  - typing_extensions=4.6.3=py38hecd8cb5_0
+  - wheel=0.38.4=py38hecd8cb5_0
+  - xz=5.4.2=h6c40b1e_0
+  - zeromq=4.3.4=h23ab428_0
+  - zlib=1.2.13=h4dc903c_0
+  - pip:
+    - aiohttp==3.8.4
+    - aiosignal==1.3.1
+    - annomate==1.0.0
+    - ansi2html==1.8.0
+    - anyio==3.7.0
+    - appnope==0.1.3
+    - argon2-cffi==21.3.0
+    - argon2-cffi-bindings==21.2.0
+    - arrow==1.2.3
+    - asttokens==2.2.1
+    - async-lru==2.0.2
+    - async-timeout==4.0.2
+    - attrs==23.1.0
+    - babel==2.12.1
+    - beautifulsoup4==4.12.2
+    - bleach==6.0.0
+    - blinker==1.8.2
+    - cachetools==5.3.1
+    - certifi==2023.5.7
+    - cffi==1.15.1
+    - charset-normalizer==3.1.0
+    - click==8.1.3
+    - cnv-suite==0.1.2
+    - comm==0.1.3
+    - cycler==0.11.0
+    - dash==2.17.0
+    - dash-bootstrap-components==1.4.1
+    - dash-core-components==2.0.0
+    - dash-cytoscape==0.3.0
+    - dash-daq==0.5.0
+    - dash-html-components==2.0.0
+    - dash-table==5.0.0
+    - debugpy==1.6.7
+    - defusedxml==0.7.1
+    - dill==0.3.6
+    - docutils==0.20.1
+    - exceptiongroup==1.1.1
+    - executing==1.2.0
+    - fastjsonschema==2.17.1
+    - flask==3.0.3
+    - fonttools==4.40.0
+    - fqdn==1.5.1
+    - frozendict==2.3.8
+    - frozenlist==1.3.3
+    - fsspec==2023.6.0
+    - gcsfs==2023.6.0
+    - google-api-core==2.11.1
+    - google-auth==2.21.0
+    - google-auth-oauthlib==1.0.0
+    - google-cloud-core==2.3.2
+    - google-cloud-storage==2.9.0
+    - google-crc32c==1.5.0
+    - google-resumable-media==2.5.0
+    - googleapis-common-protos==1.59.1
+    - hound==1.0.1
+    - idna==3.4
+    - importlib-metadata==6.7.0
+    - importlib-resources==5.12.0
+    - iniconfig==2.0.0
+    - intervaltree==3.1.0
+    - ipykernel==6.23.3
+    - ipython==8.12.2
+    - isoduration==20.11.0
+    - itsdangerous==2.1.2
+    - jaraco-classes==3.2.3
+    - jedi==0.18.2
+    - jinja2==3.1.2
+    - json5==0.9.14
+    - jsonpointer==2.4
+    - jsonschema==4.17.3
+    - jupyter-client==8.3.0
+    - jupyter-core==5.3.1
+    - jupyter-events==0.6.3
+    - jupyter-lsp==2.2.0
+    - jupyter-server==2.6.0
+    - jupyter-server-terminals==0.4.4
+    - jupyterlab==4.0.2
+    - jupyterlab-pygments==0.2.2
+    - jupyterlab-server==2.23.0
+    - kaleido==0.2.1
+    - keyring==24.2.0
+    - kiwisolver==1.4.4
+    - markdown-it-py==3.0.0
+    - markupsafe==2.1.3
+    - matplotlib==3.5.3
+    - mdurl==0.1.2
+    - mistune==3.0.1
+    - more-itertools==9.1.0
+    - multidict==6.0.4
+    - natsort==8.4.0
+    - nbclient==0.8.0
+    - nbconvert==7.6.0
+    - nbformat==5.9.0
+    - notebook-shim==0.2.3
+    - numpy==1.22.4
+    - oauthlib==3.2.2
+    - overrides==7.3.1
+    - packaging==23.1
+    - pandarallel==1.6.5
+    - pandas==1.5.3
+    - pandocfilters==1.5.0
+    - pillow==9.5.0
+    - pkginfo==1.9.6
+    - pkgutil-resolve-name==1.3.10
+    - platformdirs==3.8.0
+    - plotly==5.15.0
+    - pluggy==1.5.0
+    - prometheus-client==0.17.0
+    - prompt-toolkit==3.0.38
+    - protobuf==4.23.3
+    - psutil==5.9.5
+    - pyasn1==0.5.0
+    - pyasn1-modules==0.3.0
+    - pycparser==2.21
+    - pyparsing==3.1.0
+    - pyrsistent==0.19.3
+    - pytest==8.2.1
+    - python-json-logger==2.0.7
+    - pytz==2023.3
+    - pyyaml==6.0
+    - readme-renderer==40.0
+    - requests==2.31.0
+    - requests-oauthlib==1.3.1
+    - requests-toolbelt==1.0.0
+    - retrying==1.3.4
+    - rfc3339-validator==0.1.4
+    - rfc3986==2.0.0
+    - rfc3986-validator==0.1.1
+    - rich==13.4.2
+    - rsa==4.9
+    - scipy==1.8.1
+    - send2trash==1.8.2
+    - sniffio==1.3.0
+    - sortedcontainers==2.4.0
+    - soupsieve==2.4.1
+    - stack-data==0.6.2
+    - tabulate==0.9.0
+    - tenacity==8.2.2
+    - terminado==0.17.1
+    - tinycss2==1.2.1
+    - tomli==2.0.1
+    - tornado==6.3.2
+    - tqdm==4.65.0
+    - traitlets==5.9.0
+    - twine==4.0.2
+    - uri-template==1.3.0
+    - urllib3==1.26.16
+    - wcwidth==0.2.6
+    - webcolors==1.13
+    - webencodings==0.5.1
+    - websocket-client==1.6.1
+    - werkzeug==3.0.3
+    - yarl==1.9.2
+    - zipp==3.15.0
+prefix: /Users/cchu/opt/anaconda3/envs/clean_annomate_env
diff --git a/deploy.sh b/deploy.sh
new file mode 100755
index 0000000..f619707
--- /dev/null
+++ b/deploy.sh
@@ -0,0 +1,30 @@
+#!/bin/bash
+
+# method name
+METHOD_NAME="annomate"
+
+# docker build parameters (most likely, the defaults are OK)
+EXTRA_DOCKER_BUILD_ARGS=""
+DOCKER_BUILD_PATH="."
+
+#
+# DO NOT EDIT PAST THIS POINT UNLESS YOU HAVE VERY GOOD REASON TO DO SO
+#
+
+# automatically create method version number
+NCOMMIT=$(git rev-list --count HEAD)
+BRANCH=$(git rev-parse --abbrev-ref HEAD)
+
+if [[ $BRANCH == "master" ]]; then
+	BRANCH=""
+else
+	BRANCH+="_"
+fi
+VERSION=${BRANCH}v${NCOMMIT}
+
+# build method docker
+docker build -t getzlab/$METHOD_NAME:${VERSION} ${EXTRA_DOCKER_BUILD_ARGS} ${DOCKER_BUILD_PATH}
+
+# push method docker
+docker tag getzlab/$METHOD_NAME:${VERSION} gcr.io/broad-getzlab-workflows/$METHOD_NAME:${VERSION}
+docker push gcr.io/broad-getzlab-workflows/$METHOD_NAME:${VERSION}
\ No newline at end of file
diff --git a/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb b/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb
index 9bb74af..6923cc6 100644
--- a/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb
+++ b/tutorial_notebooks/Intro_to_AnnoMate_Reviewers.ipynb
@@ -50,6 +50,21 @@
     "1. Install package: `pip install -e .`"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "e637c8ae-2ca2-487a-9522-de96383ae80f",
+   "metadata": {},
+   "source": [
+    "Running from docker container: You need to open at least 2 ports if you are using Mac or Windows:\n",
+    "1. A port to open jupyter lab (in this case, `<jupyter_port>`)\n",
+    "2. A port to open the dash app (`<dash_port>`)\n",
+    "```\n",
+    "docker run -it -p <jupyter_port>:<jupyter_port> -p <dash_port>:<dash_port> gcr.io/broad-getzlab-workflows/annomate:latest\n",
+    "cd AnnoMate/tutorial_notebooks\n",
+    "jupyter lab --ip 0.0.0.0 --port <jupyter_port> --no-browser --allow-root\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "6429e242-eb91-4419-ba41-3b63e6f1ae6d",
@@ -204,7 +219,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 53,
+   "execution_count": 4,
    "id": "8933f1a7",
    "metadata": {},
    "outputs": [],
@@ -474,10 +489,19 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 7,
    "id": "f7fd1f9d-d9f1-44c4-b9f9-d20d667a3d7e",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewDataInterface.py:41: UserWarning: Loading existing data pkl file\n",
+      "  warnings.warn(f\"Loading existing data pkl file\")\n"
+     ]
+    }
+   ],
    "source": [
     "output_pkl_path = './example_reviewer_data'\n",
     "my_reviewer.set_review_data(data_path=output_pkl_path, \n",
@@ -509,7 +533,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 8,
    "id": "32ab23ca-c27d-4c0d-9149-08c24b244d26",
    "metadata": {},
    "outputs": [
@@ -519,7 +543,7 @@
        "dict_keys(['index', 'description', 'annot_col_config_dict', 'annot_df', 'history_df', 'df'])"
       ]
      },
-     "execution_count": 17,
+     "execution_count": 8,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -530,7 +554,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 9,
    "id": "161965c4",
    "metadata": {},
    "outputs": [
@@ -656,7 +680,7 @@
        "sample_4   Testing preprocessing  "
       ]
      },
-     "execution_count": 18,
+     "execution_count": 9,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -679,7 +703,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 10,
    "id": "cdc0779a",
    "metadata": {},
    "outputs": [],
@@ -700,7 +724,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
+   "execution_count": 11,
    "id": "2875d634",
    "metadata": {},
    "outputs": [],
@@ -711,20 +735,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
+   "execution_count": 12,
    "id": "6f611126-58d6-48b7-a056-01737bc2b263",
    "metadata": {},
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewerTemplate.py:477: UserWarning:\n",
-      "\n",
-      "Data is not frozen. Annotations will not be saved. Please freeze data in the dashboard to save annotations.\n",
-      "\n"
-     ]
-    },
     {
      "data": {
       "text/html": [
@@ -754,42 +768,42 @@
        "    <tr>\n",
        "      <th>sample_0</th>\n",
        "      <td></td>\n",
-       "      <td></td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_1</th>\n",
        "      <td></td>\n",
-       "      <td></td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_2</th>\n",
-       "      <td></td>\n",
-       "      <td></td>\n",
+       "      <td>Nothing to say</td>\n",
+       "      <td>Keep</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_3</th>\n",
        "      <td></td>\n",
-       "      <td></td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_4</th>\n",
-       "      <td></td>\n",
-       "      <td></td>\n",
+       "      <td>Not good</td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "</div>"
       ],
       "text/plain": [
-       "         Notes Flag\n",
-       "sample_0           \n",
-       "sample_1           \n",
-       "sample_2           \n",
-       "sample_3           \n",
-       "sample_4           "
+       "                   Notes    Flag\n",
+       "sample_0                  Remove\n",
+       "sample_1                  Remove\n",
+       "sample_2  Nothing to say    Keep\n",
+       "sample_3                  Remove\n",
+       "sample_4        Not good  Remove"
       ]
      },
-     "execution_count": 21,
+     "execution_count": 12,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -800,20 +814,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
+   "execution_count": 13,
    "id": "2be96b08-e7c6-477a-812b-a3a7c39d9128",
    "metadata": {},
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewerTemplate.py:485: UserWarning:\n",
-      "\n",
-      "Data is not frozen. History will not be saved. Please freeze data in the dashboard to save history.\n",
-      "\n"
-     ]
-    },
     {
      "data": {
       "text/html": [
@@ -843,17 +847,67 @@
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>sample_0</td>\n",
+       "      <td>2024-05-29 11:21:53.051037</td>\n",
+       "      <td>./example_reviewer_data/data.pkl</td>\n",
+       "      <td></td>\n",
+       "      <td>Remove</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>sample_1</td>\n",
+       "      <td>2024-05-29 11:21:56.466574</td>\n",
+       "      <td>./example_reviewer_data/data.pkl</td>\n",
+       "      <td></td>\n",
+       "      <td>Remove</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>sample_2</td>\n",
+       "      <td>2024-05-29 11:22:02.213915</td>\n",
+       "      <td>./example_reviewer_data/data.pkl</td>\n",
+       "      <td>Nothing to say</td>\n",
+       "      <td>Keep</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>sample_3</td>\n",
+       "      <td>2024-05-29 11:22:05.466076</td>\n",
+       "      <td>./example_reviewer_data/data.pkl</td>\n",
+       "      <td></td>\n",
+       "      <td>Remove</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>sample_4</td>\n",
+       "      <td>2024-05-29 11:22:12.998607</td>\n",
+       "      <td>./example_reviewer_data/data.pkl</td>\n",
+       "      <td>Not good</td>\n",
+       "      <td>Remove</td>\n",
+       "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "</div>"
       ],
       "text/plain": [
-       "Empty DataFrame\n",
-       "Columns: [index, timestamp, source_data_fn, Notes, Flag]\n",
-       "Index: []"
+       "      index                  timestamp                    source_data_fn  \\\n",
+       "0  sample_0 2024-05-29 11:21:53.051037  ./example_reviewer_data/data.pkl   \n",
+       "0  sample_1 2024-05-29 11:21:56.466574  ./example_reviewer_data/data.pkl   \n",
+       "0  sample_2 2024-05-29 11:22:02.213915  ./example_reviewer_data/data.pkl   \n",
+       "0  sample_3 2024-05-29 11:22:05.466076  ./example_reviewer_data/data.pkl   \n",
+       "0  sample_4 2024-05-29 11:22:12.998607  ./example_reviewer_data/data.pkl   \n",
+       "\n",
+       "            Notes    Flag  \n",
+       "0                  Remove  \n",
+       "0                  Remove  \n",
+       "0  Nothing to say    Keep  \n",
+       "0                  Remove  \n",
+       "0        Not good  Remove  "
       ]
      },
-     "execution_count": 22,
+     "execution_count": 13,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -876,7 +930,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 23,
+   "execution_count": 14,
    "id": "b00fd0cf",
    "metadata": {},
    "outputs": [
@@ -889,16 +943,6 @@
       "Dash app running on http://0.0.0.0:8055/\n"
      ]
     },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
-      "\n",
-      "You are in test mode. Your data will not be saved.\n",
-      "\n"
-     ]
-    },
     {
      "data": {
       "application/javascript": [
@@ -923,7 +967,12 @@
    "source": [
     "> **If you are using a VM:** Make sure to open a ssh tunnel to the specified port number\n",
     "> - `~C` to open the ssh prompt (will look like `ssh>`)\n",
-    "> - Type and enter `-L <port number>:localhost:<port number>`"
+    "> - Type and enter `-L <port number>:localhost:<port number>`\n",
+    "\n",
+    "> **If you are running from a docker container:** Make sure to open the port for accessing the dash app (`reviewer.run(port=<new_port_number>, ...)`) when you open the container\n",
+    "> - `docker run -it -p <jupyter_port>:<jupyter_port> -p <dash_port>:<dash_port> gcr.io/broad-getzlab-workflows/annomate:latest`\n",
+    "> - You can add more ports by adding more `-p <port_number>:<port_number>` arguments\n",
+    "> - If you are on Linux, you can replace `-p` arguments with `--network host`"
    ]
   },
   {
@@ -949,20 +998,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 24,
+   "execution_count": 15,
    "id": "8bff62a2",
    "metadata": {},
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewerTemplate.py:477: UserWarning:\n",
-      "\n",
-      "Data is not frozen. Annotations will not be saved. Please freeze data in the dashboard to save annotations.\n",
-      "\n"
-     ]
-    },
     {
      "data": {
       "text/html": [
@@ -1027,7 +1066,7 @@
        "sample_4        Not good  Remove"
       ]
      },
-     "execution_count": 24,
+     "execution_count": 15,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1039,20 +1078,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 25,
+   "execution_count": 16,
    "id": "852bf6fa-9016-493a-89e9-cfa29e37d9c1",
    "metadata": {},
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewerTemplate.py:485: UserWarning:\n",
-      "\n",
-      "Data is not frozen. History will not be saved. Please freeze data in the dashboard to save history.\n",
-      "\n"
-     ]
-    },
     {
      "data": {
       "text/html": [
@@ -1142,7 +1171,7 @@
        "0        Not good  Remove  "
       ]
      },
-     "execution_count": 25,
+     "execution_count": 16,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1191,7 +1220,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 26,
+   "execution_count": 17,
    "id": "725efe31",
    "metadata": {},
    "outputs": [
@@ -1199,17 +1228,8 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Export directory will be data/example_reviewer_data/2024-05-29. Nothing exported yet.\n"
-     ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewerTemplate.py:513: UserWarning:\n",
-      "\n",
-      "Directory data/example_reviewer_data/2024-05-29 already exists\n",
-      "\n"
+      "Making new directory data/example_reviewer_data/2024-06-03\n",
+      "Export directory will be data/example_reviewer_data/2024-06-03. Nothing exported yet.\n"
      ]
     }
    ],
@@ -1223,7 +1243,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 27,
+   "execution_count": 18,
    "id": "e26441c4-57df-4d06-b8cb-915f97dc9c56",
    "metadata": {},
    "outputs": [
@@ -1234,10 +1254,20 @@
       "index is not a dataframe. Not exporting.\n",
       "description is not a dataframe. Not exporting.\n",
       "annot_col_config_dict is not a dataframe. Not exporting.\n",
-      "Saving annot_df to data/example_reviewer_data/2024-05-29/annot_df.tsv\n",
-      "Saving history_df to data/example_reviewer_data/2024-05-29/history_df.tsv\n",
-      "Saving df to data/example_reviewer_data/2024-05-29/df.tsv\n",
-      "Exported to data/example_reviewer_data/2024-05-29\n"
+      "Saving annot_df to data/example_reviewer_data/2024-06-03/annot_df.tsv\n",
+      "Saving history_df to data/example_reviewer_data/2024-06-03/history_df.tsv\n",
+      "Saving df to data/example_reviewer_data/2024-06-03/df.tsv\n",
+      "Exported to data/example_reviewer_data/2024-06-03\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewerTemplate.py:513: UserWarning:\n",
+      "\n",
+      "Directory data/example_reviewer_data/2024-06-03 already exists\n",
+      "\n"
      ]
     }
    ],
@@ -1270,7 +1300,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 28,
+   "execution_count": 19,
    "id": "cec30f2c",
    "metadata": {},
    "outputs": [],
@@ -1290,7 +1320,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 29,
+   "execution_count": 20,
    "id": "1cf5f2b7",
    "metadata": {},
    "outputs": [
@@ -1339,7 +1369,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 30,
+   "execution_count": 21,
    "id": "bcaac9d1",
    "metadata": {},
    "outputs": [
@@ -1365,7 +1395,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 32,
+   "execution_count": 22,
    "id": "386d24d6-311c-4678-9832-503aadd8e058",
    "metadata": {},
    "outputs": [
@@ -1373,7 +1403,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewerTemplate.py:477: UserWarning:\n",
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewerTemplate.py:477: UserWarning:\n",
       "\n",
       "Data is not frozen. Annotations will not be saved. Please freeze data in the dashboard to save annotations.\n",
       "\n"
@@ -1400,44 +1430,44 @@
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
-       "      <th>Flag</th>\n",
        "      <th>Notes</th>\n",
+       "      <th>Flag</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>sample_0</th>\n",
-       "      <td>Remove</td>\n",
        "      <td></td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_1</th>\n",
-       "      <td>Remove</td>\n",
        "      <td></td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_3</th>\n",
-       "      <td>Remove</td>\n",
        "      <td></td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_4</th>\n",
-       "      <td>Remove</td>\n",
        "      <td>Not good</td>\n",
+       "      <td>Remove</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
        "</div>"
       ],
       "text/plain": [
-       "            Flag     Notes\n",
-       "sample_0  Remove          \n",
-       "sample_1  Remove          \n",
-       "sample_3  Remove          \n",
-       "sample_4  Remove  Not good"
+       "             Notes    Flag\n",
+       "sample_0            Remove\n",
+       "sample_1            Remove\n",
+       "sample_3            Remove\n",
+       "sample_4  Not good  Remove"
       ]
      },
-     "execution_count": 32,
+     "execution_count": 22,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1462,7 +1492,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 33,
+   "execution_count": 23,
    "id": "1e23f569",
    "metadata": {},
    "outputs": [],
@@ -1485,7 +1515,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 34,
+   "execution_count": 24,
    "id": "f44d22bb",
    "metadata": {},
    "outputs": [
@@ -1545,7 +1575,7 @@
        "2  Anthracycline  remission       203"
       ]
      },
-     "execution_count": 34,
+     "execution_count": 24,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1556,7 +1586,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 35,
+   "execution_count": 25,
    "id": "9fd80758",
    "metadata": {},
    "outputs": [],
@@ -1596,7 +1626,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 36,
+   "execution_count": 26,
    "id": "b89c3fea-2afd-4075-a473-e8a85fcc7f5c",
    "metadata": {},
    "outputs": [],
@@ -1608,7 +1638,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 37,
+   "execution_count": 27,
    "id": "6153a01e-65d4-4300-a06e-b005ef474990",
    "metadata": {},
    "outputs": [],
@@ -1618,7 +1648,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 38,
+   "execution_count": 28,
    "id": "9849c027-aa61-45e5-a70d-76abe4142cd9",
    "metadata": {},
    "outputs": [],
@@ -1666,7 +1696,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 39,
+   "execution_count": 29,
    "id": "3f91079d-0084-4abf-a4b0-ecb292e65d7f",
    "metadata": {},
    "outputs": [],
@@ -1744,7 +1774,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 40,
+   "execution_count": 30,
    "id": "7d013fe3-fd80-49de-818f-29a2748722b7",
    "metadata": {},
    "outputs": [
@@ -1754,7 +1784,7 @@
        "True"
       ]
      },
-     "execution_count": 40,
+     "execution_count": 30,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1775,7 +1805,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 41,
+   "execution_count": 31,
    "id": "f1e83bc2-f597-4e80-82f4-558265058096",
    "metadata": {},
    "outputs": [
@@ -1785,7 +1815,7 @@
        "False"
       ]
      },
-     "execution_count": 41,
+     "execution_count": 31,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1826,7 +1856,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 42,
+   "execution_count": 32,
    "id": "67cb42cd",
    "metadata": {},
    "outputs": [],
@@ -1849,7 +1879,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 43,
+   "execution_count": 33,
    "id": "7871969b",
    "metadata": {},
    "outputs": [
@@ -1858,7 +1888,6 @@
      "output_type": "stream",
      "text": [
       "Setting auto_export_path to ./example_reviewer_data_removed/data.auto_export\n",
-      "Making directory ./example_reviewer_data_removed/data.auto_export for auto exporting.\n",
       "Using ./example_reviewer_data_removed/data.auto_export for auto exporting.\n",
       "Dash app running on http://0.0.0.0:8052/\n"
      ]
@@ -1867,7 +1896,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
       "\n",
       "You are in test mode. Your data will not be saved.\n",
       "\n"
@@ -1908,7 +1937,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 52,
+   "execution_count": 34,
    "id": "b7630df3-40b5-4e8a-9638-cdf4cd59022b",
    "metadata": {},
    "outputs": [
@@ -1917,26 +1946,20 @@
      "output_type": "stream",
      "text": [
       "Setting auto_export_path to ./example_reviewer_data_removed/data.auto_export\n",
-      "Using ./example_reviewer_data_removed/data.auto_export for auto exporting.\n"
+      "Using ./example_reviewer_data_removed/data.auto_export for auto exporting.\n",
+      "Dash app running on http://0.0.0.0:8052/\n"
      ]
     },
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
       "\n",
       "You are in test mode. Your data will not be saved.\n",
       "\n"
      ]
     },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Dash app running on http://0.0.0.0:8052/\n"
-     ]
-    },
     {
      "data": {
       "application/javascript": [
@@ -1961,7 +1984,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 44,
+   "execution_count": 35,
    "id": "83dee02a-8415-4a1c-9b85-b2598883a0c7",
    "metadata": {},
    "outputs": [
@@ -1969,7 +1992,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewerTemplate.py:477: UserWarning:\n",
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewerTemplate.py:477: UserWarning:\n",
       "\n",
       "Data is not frozen. Annotations will not be saved. Please freeze data in the dashboard to save annotations.\n",
       "\n"
@@ -1996,8 +2019,8 @@
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
-       "      <th>Flag</th>\n",
        "      <th>Notes</th>\n",
+       "      <th>Flag</th>\n",
        "      <th>Histogram color</th>\n",
        "      <th>A Number</th>\n",
        "      <th>A radioitem</th>\n",
@@ -2010,44 +2033,44 @@
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>sample_0</th>\n",
-       "      <td>Report</td>\n",
+       "      <td></td>\n",
+       "      <td>Remove</td>\n",
+       "      <td></td>\n",
+       "      <td>NaN</td>\n",
+       "      <td></td>\n",
        "      <td></td>\n",
        "      <td></td>\n",
-       "      <td>1.0</td>\n",
-       "      <td>Option b</td>\n",
-       "      <td>[Option 1, Option 3]</td>\n",
        "      <td></td>\n",
-       "      <td>[Option y, Option z]</td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_1</th>\n",
+       "      <td></td>\n",
        "      <td>Remove</td>\n",
        "      <td></td>\n",
-       "      <td>red</td>\n",
-       "      <td>10.0</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td></td>\n",
+       "      <td></td>\n",
+       "      <td></td>\n",
        "      <td></td>\n",
-       "      <td>[Option 1]</td>\n",
-       "      <td>Option C</td>\n",
-       "      <td>[Option x]</td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_3</th>\n",
-       "      <td>Keep</td>\n",
        "      <td></td>\n",
+       "      <td>Remove</td>\n",
        "      <td></td>\n",
        "      <td>NaN</td>\n",
-       "      <td>Option a</td>\n",
-       "      <td>[Option 1]</td>\n",
-       "      <td>Option B</td>\n",
-       "      <td>[]</td>\n",
+       "      <td></td>\n",
+       "      <td></td>\n",
+       "      <td></td>\n",
+       "      <td></td>\n",
        "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>sample_4</th>\n",
-       "      <td>Remove</td>\n",
        "      <td>Not good</td>\n",
+       "      <td>Remove</td>\n",
        "      <td></td>\n",
        "      <td>NaN</td>\n",
        "      <td></td>\n",
@@ -2061,17 +2084,17 @@
        "</div>"
       ],
       "text/plain": [
-       "            Flag     Notes Histogram color  A Number A radioitem  \\\n",
-       "sample_0  Report                                 1.0    Option b   \n",
-       "sample_1  Remove                       red      10.0               \n",
-       "sample_3    Keep                                 NaN    Option a   \n",
-       "sample_4  Remove  Not good                       NaN               \n",
+       "             Notes    Flag Histogram color  A Number A radioitem A checklist  \\\n",
+       "sample_0            Remove                       NaN                           \n",
+       "sample_1            Remove                       NaN                           \n",
+       "sample_3            Remove                       NaN                           \n",
+       "sample_4  Not good  Remove                       NaN                           \n",
        "\n",
-       "                   A checklist A dropdown menu A multi-select dropdown menu  \\\n",
-       "sample_0  [Option 1, Option 3]                         [Option y, Option z]   \n",
-       "sample_1            [Option 1]        Option C                   [Option x]   \n",
-       "sample_3            [Option 1]        Option B                           []   \n",
-       "sample_4                                                                      \n",
+       "         A dropdown menu A multi-select dropdown menu  \\\n",
+       "sample_0                                                \n",
+       "sample_1                                                \n",
+       "sample_3                                                \n",
+       "sample_4                                                \n",
        "\n",
        "          Test removing annotation  \n",
        "sample_0                       NaN  \n",
@@ -2080,7 +2103,7 @@
        "sample_4                       NaN  "
       ]
      },
-     "execution_count": 44,
+     "execution_count": 35,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2117,7 +2140,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 45,
+   "execution_count": 36,
    "id": "d05f523a-a3d6-4ae0-8279-c9d115a2ecf8",
    "metadata": {},
    "outputs": [
@@ -2126,25 +2149,19 @@
      "output_type": "stream",
      "text": [
       "Setting auto_export_path to ./example_reviewer_data_removed/data.auto_export\n",
-      "Using ./example_reviewer_data_removed/data.auto_export for auto exporting.\n"
+      "Using ./example_reviewer_data_removed/data.auto_export for auto exporting.\n",
+      "Dash app running on http://0.0.0.0:8052/\n"
      ]
     },
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
       "\n",
       "You are in test mode. Your data will not be saved.\n",
       "\n"
      ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Dash app running on http://0.0.0.0:8052/\n"
-     ]
     }
    ],
    "source": [
@@ -2174,7 +2191,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 46,
+   "execution_count": 37,
    "id": "8e272002-5cb0-4017-9189-385ba1fe47e4",
    "metadata": {},
    "outputs": [
@@ -2184,7 +2201,7 @@
        "odict_keys(['sample-info-component', 'Maf file', 'Mut vafs', 'Treatment file'])"
       ]
      },
-     "execution_count": 46,
+     "execution_count": 37,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -2203,7 +2220,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 51,
+   "execution_count": 38,
    "id": "068c61f2-ecd0-467c-a9e9-6894fbf16730",
    "metadata": {},
    "outputs": [
@@ -2212,19 +2229,25 @@
      "output_type": "stream",
      "text": [
       "Setting auto_export_path to ./example_reviewer_data_removed/data.auto_export\n",
-      "Using ./example_reviewer_data_removed/data.auto_export for auto exporting.\n",
-      "Dash app running on http://0.0.0.0:8052/\n"
+      "Using ./example_reviewer_data_removed/data.auto_export for auto exporting.\n"
      ]
     },
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/cchu/Desktop/Methods/AnnoMate/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
+      "/Users/cchu/opt/anaconda3/envs/test/lib/python3.8/site-packages/AnnoMate/ReviewDataApp.py:738: UserWarning:\n",
       "\n",
       "You are in test mode. Your data will not be saved.\n",
       "\n"
      ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Dash app running on http://0.0.0.0:8052/\n"
+     ]
     }
    ],
    "source": [
@@ -2262,9 +2285,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python [conda env:test_annomate_install_env]",
+   "display_name": "Python [conda env:test]",
    "language": "python",
-   "name": "conda-env-test_annomate_install_env-py"
+   "name": "conda-env-test-py"
   },
   "language_info": {
    "codemirror_mode": {


Test patch:
diff --git a/tests/test_example_reviewer.py b/tests/test_example_reviewer.py
index 3af375a..8711889 100644
--- a/tests/test_example_reviewer.py
+++ b/tests/test_example_reviewer.py
@@ -2,7 +2,7 @@
 from AnnoMate.Reviewers.ExampleReviewer import ExampleReviewer
 
 def test_reviewer():
-    fn = '../tutorial_notebooks/example_data/AnnoMate_Tutorial/data_to_review_example.tsv'
+    fn = 'tutorial_notebooks/example_data/AnnoMate_Tutorial/data_to_review_example.tsv'
     df = pd.read_csv(fn, sep='\t')
     df = df.set_index('sample_id')
     assert df.shape[0] > 0