Repository: Significant-Gravitas/AutoGPT
Pull request: Option to disable working directory restrictions
### Background
It is constantly unable to access files due to workspace working directory restrictions. An option is needed to remove the restriction on the working directory.

### Changes
- Added new environment variable `WORKING_DIRECTORY_RESTRICTED` that is set to True by default, setting to False will remove the restriction.
- Modfied `execute_code.py` to use `safe_join` from `file_operations.py` instead so that the environment variable can be referenced once in `file_operations.py` only.

### Documentation
Above.

### Test Plan
Tested by making sure it was able to access files outside of the working directory with it set to false.

### PR Quality Checklist
- [x] My pull request is atomic and focuses on a single change.
- [x] I have thoroughly tested my changes with multiple different prompts.
- [x] I have considered potential risks and mitigations for my changes.
- [x] I have documented my changes clearly and comprehensively.
- [x] I have not snuck in any "extra" small tweaks changes

Problem statement:
Ability to Navigate Outside Working Directory.
Currently, the best workaround is bringing the desired context into the working directory, but implementing the ability to crawl through files more gracefully would be great. Perhaps for fully autonomous users, this could be an opt-in or out type thing.

SYSTEM:  Command read_file returned: Error: Attempted to access outside of working directory.

(This project rocks by the way, great work!)

Add option to allow access outside of working directory
### Duplicates

- [X] I have searched the existing issues

### Summary 💡

Yeah, I know, probably not always ideal, but having this thing limited to a working directory that it can't seem to access half the time is not ideal.  It would be good to have an option to bypass this safety with an environment variable that can allow people to give it the option to read and write files anywhere it wants.

I personally run Auto-GPT in an Ubuntu VM that I set up for the sole purpose of letting it break itself if it needs to.  It isn't ideal for everyone to have this on, but it would be helpful to have as an option.

### Examples 🌈

```
Executing command 'ls ~/Agent' in working directory '/home/josh/Auto-GPT/autogpt/auto_gpt_workspace'
SYSTEM:  Command execute_shell returned: STDOUT: b'' STDERR: b"ls: cannot access '/home/josh/Agent': No such file or directory\n"
```

### Motivation 🔦

Easier to allow it to work with existing files than making duplicates in its workspace.

Hints from issue discussion:
Yeah, it's an interesting option.

I think the major concern from that though would be that it could, and would easily delete/overwrite files.

I have been playing about with the same approach as you, cloning the repo into the AI's working directory! 😆
Thank you so much for the kind words 😊 I'm glad you're enjoying it!
What exactly is the working directory for this? I have put my file in the same folder as the main.py file and it still gives me an error saying it's outside the Working Directory. I am working on an M1 Mac, is there other permissions I need to grant for it to have access?

> What exactly is the working directory for this? I have put my file in the same folder as the main.py file and it still gives me an error saying it's outside the Working Directory. I am working on an M1 Mac, is there other permissions I need to grant for it to have access?

same issue on my M1 Mac
I tried making a link to the workspace from inside the AutoGPT dir:

```bash
~/github/Auto-GPT on master! ⌚ 9:42:38
$ ln -s ~/code/<my_repo>
```

(Note: From my use, the search_files always returns an empty list, regardless of whether accessing an inode actually under AutoGPT or just a link.)


> What exactly is the working directory for this? I have put my file in the same folder as the main.py file and it still gives me an error saying it's outside the Working Directory. I am working on an M1 Mac, is there other permissions I need to grant for it to have access?

It's the "auto_gpt_workspace" folder.

> It's the "auto_gpt_workspace" folder.

So, putting the file that you want it to read, inside of "auto_gpt_workspace" should let the program access it?
I have my files in project folders. it is really cumbersome to move them around.
Also another big issue is that files have references to other files in the project folder, so auto-gpt should be optimised to read the referenced files as well, isn't it ?
I am trying to search in the code where is this folder, but i think it should be like an option in the begining to choose the working folder .

Also, another suggestion would be to really do something about the agents like you wrote here ;)

    # Soon this will go in a folder where it remembers more stuff about the run(s)
    SAVE_FILE = os.path.join(os.path.dirname(__file__), '..', 'ai_settings.yaml')
    
    workaround is to save each yaml file as a backup
I'm still getting the message "error: attempted to access outside of working directory". I have saved a .py file to the workspace directory and have tasked auto-gpt with completing the code. I've given auto-gpt the location of the directory "C:auto_gpt\auto_gpt_workspace" for it's goals and description prompts and "C:auto_gpt" is where I run auto-gpt in powershell. With a blank .py file or a .py program template with descriptions and context it continues to have the error message. I've seen people have auto-gpt write code and I'm very excited to try it out myself.
It made it unusable for my use case, so I put in pull request #1875 to remove the working directory restrictions.  I still suggest making them into environment variables for people who care about restricting it to a working directory.

Commit message: Updating if statement on check

Patch to review:
diff --git a/autogpt/commands/execute_code.py b/autogpt/commands/execute_code.py
index 86d6c177b796..0e824aed362a 100644
--- a/autogpt/commands/execute_code.py
+++ b/autogpt/commands/execute_code.py
@@ -2,7 +2,7 @@
 import os
 from pathlib import Path
 import subprocess
-
+from autogpt.commands.file_operations import safe_join
 import docker
 from docker.errors import ImageNotFound
 
@@ -100,9 +100,7 @@ def execute_shell(command_line: str) -> str:
     """
     current_dir = os.getcwd()
     # Change dir into workspace if necessary
-    if str(WORKING_DIRECTORY) not in current_dir:
-        work_dir = os.path.join(os.getcwd(), WORKING_DIRECTORY)
-        os.chdir(work_dir)
+    os.chdir(safe_join(current_dir, WORKING_DIRECTORY))
 
     print(f"Executing command '{command_line}' in working directory '{os.getcwd()}'")
 
diff --git a/autogpt/commands/file_operations.py b/autogpt/commands/file_operations.py
index d02b125a79ed..08298abe4c0d 100644
--- a/autogpt/commands/file_operations.py
+++ b/autogpt/commands/file_operations.py
@@ -3,7 +3,8 @@
 import os.path
 from pathlib import Path
 from typing import Generator, List
-
+from autogpt.config import Config
+CFG = Config()
 # Set a dedicated folder for file I/O
 WORKING_DIRECTORY = Path(os.getcwd()) / "auto_gpt_workspace"
 
@@ -58,13 +59,13 @@ def safe_join(base: str, *paths) -> str:
     Returns:
         str: The joined path
     """
-    new_path = os.path.join(base, *paths)
-    norm_new_path = os.path.normpath(new_path)
-
-    if os.path.commonprefix([base, norm_new_path]) != base:
-        raise ValueError("Attempted to access outside of working directory.")
-
-    return norm_new_path
+    if str(CFG.working_directory_restricted).lower() == "true":
+        new_path = os.path.normpath(os.path.join(base, *paths))
+        if os.path.commonprefix([base, new_path]) != base:
+            raise ValueError("Attempted to access outside of working directory.")
+    else:
+        new_path = os.path.normpath(os.path.join("/", *paths))
+    return new_path
 
 
 def split_file(
diff --git a/autogpt/config/config.py b/autogpt/config/config.py
index 22da52b047e7..9e242d998cfa 100644
--- a/autogpt/config/config.py
+++ b/autogpt/config/config.py
@@ -40,7 +40,9 @@ def __init__(self) -> None:
         self.execute_local_commands = (
             os.getenv("EXECUTE_LOCAL_COMMANDS", "False") == "True"
         )
-
+        self.working_directory_restricted = (
+            os.getenv("WORKING_DIRECTORY_RESTRICTED", "True") == "True"
+        )
         if self.use_azure:
             self.load_azure_config()
             openai.api_type = self.openai_api_type


Code review comments:
diff_hunk: @@ -100,9 +100,7 @@ def execute_shell(command_line: str) -> str:
     """
     current_dir = os.getcwd()
     # Change dir into workspace if necessary
-    if str(WORKING_DIRECTORY) not in current_dir:
-        work_dir = os.path.join(os.getcwd(), WORKING_DIRECTORY)
-        os.chdir(work_dir)
+    os.chdir(safe_join(current_dir, WORKING_DIRECTORY))
path: autogpt/commands/execute_code.py
text: @user1:
Don't forget to mention this fix in the PR description. It's out of scope for the current description.

@author:
Updated.

@user1:
On second note.. this fix could rightfully get its own PR, since it fixes workspace path determination for `execute_shell`, which is currently broken. Allowing access outside of the workspace is a separate issue.

@author:
`safe_join` is essentially doing the same thing.