Repository: PrefectHQ/prefect

Issue:
Error while deploying flow: FileNotFoundError(2, "No such file or directory: 'prefect'")
## Description

Getting "No such file or directory: 'prefect'" error when running a simple task from the web UI. (Note there are no directories within my project named "prefect")


## Expected Behavior

Expected the task to run and display "success" on the web page.


## Reproduction



    from prefect import task, Flow

    @task
    def the_simple():
        print('hello')


    if __name__ == '__main__':

        with Flow('the-simplest') as flow:
            the_simple()

        flow.register('test-amazon')




## Environment


Operating system is Amazon Linux 2.
Have also tried running on Kali linux, and the same error shows up.

`$ env/bin/prefect diagnostics
{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "Linux-4.14.203-156.332.amzn2.x86_64-x86_64-with-glibc2.2.5",
    "prefect_backend": "server",
    "prefect_version": "0.13.18",
    "python_version": "3.7.9"
  }
}`



Discussion:
> `"No such file or directory: 'prefect'" `

This is probably due to `prefect` (the library) being missing/not on-path in your flow run environment.

What prefect agent are you running? How are you deploying it?
@jcrist The prefect agent is running locally. From a terminal, this command is run:

    env/bin/prefect agent local start
Here is a recipe to recreate the error:

From a terminal:


    mkdir v4
    cd v4
    vagrant init bento/amazonlinux-2
    vagrant up
    vagrant ssh

    # prereqs
    sudo yum install -y python3 docker gcc python3-devel

    # Start docker service
    sudo systemctl enable docker
    sudo systemctl start docker

    # Add vagrant to docker group so can use without sudo
    sudo usermod -a -G docker vagrant

    # Exit from the VM
    exit

    # Reboot the VM
    vagrant reload

    # docker-compose
    sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
    # make it executable
    sudo chmod +x /usr/local/bin/docker-compose

    # python packages
    mkdir -p ~/p/testing
    cd ~/p/testing
    python3 -m venv env
    source env/bin/activate
    pip install prefect

    # server or cloud
    prefect backend server

    # If running in Vagrant, add these network settings to your Vagrantfile:
    config.vm.network "forwarded_port", guest: 4200, host: 4200
    config.vm.network "forwarded_port", guest: 8080, host: 8080

    # window 1 (server with persistent storage)
    prefect server start --use-volume

    # window 2 (simple/local agent)
    prefect agent local start

    # Create a project
    prefect create project simple-project   

Simple Task
-----------

Save the following as simple.py


    # simple.py

    from prefect import task, Flow

    @task
    def simple():
        print('hello')


    if __name__ == '__main__':

        with Flow('simple-flow') as flow:
            simple()

        flow.register('simple-project')


Run the file in order to register the flow:

    python simple.py



Browser
-------

Open the browser to localhost:8080
Run the flow and wait for an error  
>  From a terminal, this command is run:
>
> `env/bin/prefect agent local start`

Is `prefect` not on path in your agent environment then (meaning, can you run `prefect agent local start` instead of requiring the full path to `prefect`)?
@jcrist No, `prefect` is not part of the system path that the agent sees. That is, running `prefect agent local start` returns this error:

`-bash: prefect: command not found`
@jcrist Tested this out:

`source env/bin/activate`
`prefect agent local start`

Then clicked "Quick Run" from localhost:8080, and the run-flow succeeds. 

Which part of the documentation can read to understand how this works? 
Ah, that's it then. I'll push a patch before the next release to fix this. For now the `prefect` cli is required to be on path.

> Which part of the documentation can read to understand how this works?

Glad you got things working. What are you looking for a reference on specifically? The orchestration docs are here: https://docs.prefect.io/orchestration/, but I'd be happy to point you at a specific page if needed.
@jcrist 
I guess I'm looking for the part of the documentation that says "the `prefect` cli is required to be on path." But if you're saying that's an oversight, then maybe it isn't in the docs. 

( when I make a mistake I like to find the docs that could have routed me around the mistake, so I can learn to read docs better)

Thank you for pushing a patch. Until then I'll make sure `prefect` is in the path.  
Yeah, this is just a small bug in the local agent (other agents won't have this issue).

Resolving patch:
diff --git a/src/prefect/__main__.py b/src/prefect/__main__.py
new file mode 100644
--- /dev/null
+++ b/src/prefect/__main__.py
@@ -0,0 +1,4 @@
+from .cli import cli
+
+if __name__ == "__main__":
+    cli()
diff --git a/src/prefect/agent/local/agent.py b/src/prefect/agent/local/agent.py
--- a/src/prefect/agent/local/agent.py
+++ b/src/prefect/agent/local/agent.py
@@ -157,7 +157,7 @@ def deploy_flow(self, flow_run: GraphQLResult) -> str:
         # show flow logs, these log entries will continue to stream to the users terminal
         # until these child processes exit, even if the agent has already exited.
         p = Popen(
-            ["prefect", "execute", "flow-run"],
+            [sys.executable, "-m", "prefect", "execute", "flow-run"],
             stdout=stdout,
             stderr=STDOUT,
             env=env,
diff --git a/src/prefect/engine/cloud/flow_runner.py b/src/prefect/engine/cloud/flow_runner.py
--- a/src/prefect/engine/cloud/flow_runner.py
+++ b/src/prefect/engine/cloud/flow_runner.py
@@ -1,5 +1,6 @@
 import os
 import signal
+import sys
 import threading
 from time import sleep as time_sleep
 from typing import Any, Callable, Dict, Iterable, Optional, Iterator
@@ -73,7 +74,15 @@ def _heartbeat(self) -> bool:
             # use empty string for testing purposes
             flow_run_id = prefect.context.get("flow_run_id", "")  # type: str
             self.client.update_flow_run_heartbeat(flow_run_id)
-            self.heartbeat_cmd = ["prefect", "heartbeat", "flow-run", "-i", flow_run_id]
+            self.heartbeat_cmd = [
+                sys.executable,
+                "-m",
+                "prefect",
+                "heartbeat",
+                "flow-run",
+                "-i",
+                flow_run_id,
+            ]
 
             query = {
                 "query": {


Test patch:
diff --git a/tests/agent/test_local_agent.py b/tests/agent/test_local_agent.py
--- a/tests/agent/test_local_agent.py
+++ b/tests/agent/test_local_agent.py
@@ -1,5 +1,6 @@
 import os
 import socket
+import sys
 from unittest.mock import MagicMock
 
 import pytest
@@ -446,7 +447,7 @@ def test_local_agent_heartbeat(monkeypatch, returncode, show_flow_logs, logs):
     popen = MockPopen()
     # expect a process to be called with the following command (with specified behavior)
     popen.set_command(
-        "prefect execute flow-run",
+        [sys.executable, "-m", "prefect", "execute", "flow-run"],
         stdout=b"awesome output!",
         stderr=b"blerg, eRroR!",
         returncode=returncode,
diff --git a/tests/engine/cloud/test_cloud_flow_runner.py b/tests/engine/cloud/test_cloud_flow_runner.py
--- a/tests/engine/cloud/test_cloud_flow_runner.py
+++ b/tests/engine/cloud/test_cloud_flow_runner.py
@@ -1,5 +1,6 @@
 import datetime
 import itertools
+import sys
 import time
 import threading
 from datetime import timedelta
@@ -413,7 +414,15 @@ def test_flow_runner_heartbeat_sets_command(monkeypatch, setting_available):
         res = runner._heartbeat()
 
     assert res is True
-    assert runner.heartbeat_cmd == ["prefect", "heartbeat", "flow-run", "-i", "foo"]
+    assert runner.heartbeat_cmd == [
+        sys.executable,
+        "-m",
+        "prefect",
+        "heartbeat",
+        "flow-run",
+        "-i",
+        "foo",
+    ]
 
 
 def test_flow_runner_does_not_have_heartbeat_if_disabled(monkeypatch):