diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
new file mode 100644
index 000000000..9d44afde7
--- /dev/null
+++ b/.devcontainer/devcontainer.json
@@ -0,0 +1,25 @@
+// File only needed for VSCode users to have proper Docker based interpreters
+{
+    "name": "accelerate_dev_environment",
+    "build": {
+        // ACTION NEEDED: comment/uncomment the relevant line depending on whether you are in a CPU/GPU environment
+         "dockerfile": "../docker/accelerate-cpu/Dockerfile"
+//        "dockerfile": "../docker/accelerate-gpu/Dockerfile"
+    },
+    "runArgs": [
+        // ACTION NEEDED: uncomment the next line if your local machine has GPUs available
+//        "--gpus", "all",
+        // Enable the docker container to access system resources
+        "--ipc", "host"
+    ],
+    "remoteEnv": {
+        "PYTHONPATH": "${containerEnv:PATH}:${containerWorkspaceFolder}"
+    },
+    "extensions": [
+        // Ensure we have IntelliSense in VSCode when running inside container
+        "ms-python.python"
+    ],
+    "workspaceFolder": "/workspaces/accelerate",
+    // Need git for VSCode to color code modifications. Only runs when building environment.
+    "onCreateCommand": "apt-get update && apt-get install -y git && pip install -e '.[dev]'"
+}
\ No newline at end of file
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index fcc7d9b1c..d0e142b16 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -130,6 +130,9 @@ Follow these steps to start contributing:
    it with `pip uninstall accelerate` before reinstalling it in editable
    mode with the `-e` flag.)
 
+   Alternatively, if you are using [Visual Studio Code](https://code.visualstudio.com/Download), the fastest way to get set up is by using
+   the provided Dev Container. Documentation on how to get started with dev containers is available [here](https://code.visualstudio.com/docs/remote/containers).
+
 5. Develop the features on your branch.
 
    As you work on the features, you should make sure that the test suite


Review comments: body: I think we should enable CPU by default and have the user work on the container to activate GPU.
diff_hunk: @@ -0,0 +1,25 @@
+// File only needed for VSCode users to have proper Docker based interpreters
+{
+    "name": "accelerate_dev_environment",
+    "build": {
+        // ACTION NEEDED: comment/uncomment the relevant line depending on whether you are in a CPU/GPU environment
+        // "dockerfile": "../docker/accelerate-cpu/Dockerfile"
body: Same here.
diff_hunk: @@ -0,0 +1,25 @@
+// File only needed for VSCode users to have proper Docker based interpreters
+{
+    "name": "accelerate_dev_environment",
+    "build": {
+        // ACTION NEEDED: comment/uncomment the relevant line depending on whether you are in a CPU/GPU environment
+        // "dockerfile": "../docker/accelerate-cpu/Dockerfile"
+        "dockerfile": "../docker/accelerate-gpu/Dockerfile"
+    },
+    "runArgs": [
+        // ACTION NEEDED: comment the next line if your local machine has no GPUs available
body: Sure, I was torn on this, so am happy to go the other way.
diff_hunk: @@ -0,0 +1,25 @@
+// File only needed for VSCode users to have proper Docker based interpreters
+{
+    "name": "accelerate_dev_environment",
+    "build": {
+        // ACTION NEEDED: comment/uncomment the relevant line depending on whether you are in a CPU/GPU environment
+        // "dockerfile": "../docker/accelerate-cpu/Dockerfile"

Discussion: body: _The documentation is not available anymore as the PR was closed or merged._
body: > Also, if the users change the devcontainer.json for their local use, won't those changes be pushed to GitHub automatically?

Unfortunately, VSCode doesn't (yet) support multiple dev container configurations to my knowledge, so we have to rely on people not committing dev container changes.