Repository: PrefectHQ/prefect

Issue:
`unittest` framework raises `ResourceWarning`
## Description
<!-- A clear description of the bug -->
I'm using [`unittest`](https://docs.python.org/3/library/unittest.html) framework for testing purposes and then a task is running inside a `unittest.TestCase`, a `ResourceWarning` is raised. 


## Expected Behavior
<!-- What did you expect to happen instead? -->
Run testcase without raising any warnings.




## Reproduction
<!-- A minimal example that exhibits the behavior. -->

`test.py` file:

```python
import typing
import unittest

from prefect.engine.task_runner import TaskRunner
from prefect.tasks.shell import ShellTask


class ShellTestCase(unittest.TestCase):
    def test_shell_command(self) -> typing.NoReturn:
        runner = TaskRunner(ShellTask('ls'))
        runner.run()
        self.assertTrue(True)
```

execution:

```bash
$ python -m unittest test.py 
[2020-07-20 16:13:20] INFO - prefect.TaskRunner | Task 'ShellTask': Starting task run...
/home/psimakis/.local/share/virtualenvs/data-workflows-GfPV92cZ/lib/python3.7/site-packages/prefect/utilities/tasks.py:444: ResourceWarning: unclosed file <_io.BufferedReader name=9>
  return run_method(self, *args, **kwargs)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
[2020-07-20 16:13:21] INFO - prefect.TaskRunner | Task 'ShellTask': finished task run for task with final state: 'Success'
.
----------------------------------------------------------------------
Ran 1 test in 0.026s
```




## Environment
<!-- Any additional information about your environment

Optionally run `prefect diagnostics` from the command line and paste the information here. -->
```json
{
  "config_overrides": {},
  "env_vars": [
    "PREFECT__CONTEXT__SECRETS__....",
    "PREFECT__CONTEXT__SECRETS__...."
  ],
  "system_information": {
    "platform": "Linux-5.3.0-28-generic-x86_64-with-debian-buster-sid",
    "prefect_version": "0.12.3",
    "python_version": "3.7.3"
  }
}
```


Discussion:
Hi @sp1thas! Just wanted to note that I was able to reproduce your issue, and am taking a look! Interestingly I could reproduce with your example, but not with one that didn't use ShellTask, so a possible lead here is something related to the ShellTask implementation specifically. Here is some output I got when I [turned on tracebackmalloc to report the top 10 files opened](https://docs.python.org/3/library/tracemalloc.html#display-the-top-10) that also implies the 'bad file' not being closed is from one opened by `subprocess`, which is used in the case of a ShellTask specifically:

Using the ShellTask:
```
(prefect_dev) ➜  active_issues python -m unittest issue-3002-resourcewarninginunittest.py
[2020-07-21 16:11:33] INFO - prefect.TaskRunner | Task 'ShellTask': Starting task run...
[2020-07-21 16:11:33] DEBUG - prefect.TaskRunner | Task 'ShellTask': Handling state change from Pending to Running
[2020-07-21 16:11:33] DEBUG - prefect.TaskRunner | Task 'ShellTask': Calling task.run() method...
[2020-07-21 16:11:33] DEBUG - prefect.ShellTask | __pycache__
[2020-07-21 16:11:33] DEBUG - prefect.ShellTask | issue-3002-resourcewarninginunittest.py
/Users/laura/Development/prefect/src/prefect/utilities/tasks.py:445: ResourceWarning: unclosed file <_io.BufferedReader name=8>
  return run_method(self, *args, **kwargs)
Object allocated at (most recent call last):
  File "/opt/anaconda3/envs/prefect_dev/lib/python3.8/subprocess.py", lineno 844
    self.stdout = io.open(c2pread, 'rb', bufsize)
[2020-07-21 16:11:33] DEBUG - prefect.TaskRunner | Task 'ShellTask': Handling state change from Running to Success
[2020-07-21 16:11:33] INFO - prefect.TaskRunner | Task 'ShellTask': finished task run for task with final state: 'Success'
/opt/anaconda3/envs/prefect_dev/lib/python3.8/linecache.py:137: size=205 KiB, count=2022, average=104 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/abc.py:102: size=26.7 KiB, count=240, average=114 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:203: size=11.9 KiB, count=23, average=531 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:141: size=10.9 KiB, count=100, average=112 B
/Users/laura/Development/prefect/src/prefect/engine/runner.py:48: size=4920 B, count=10, average=492 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:402: size=4608 B, count=9, average=512 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:143: size=4536 B, count=21, average=216 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:443: size=4296 B, count=75, average=57 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/tempfile.py:137: size=2656 B, count=3, average=885 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:434: size=2240 B, count=10, average=224 B
.
----------------------------------------------------------------------
Ran 1 test in 0.071s

OK
```

Using a plain jane task, NO ShellTask:
```
(prefect_dev) ➜  RandomFlows python -m unittest issue-3002-resourcewarninginunittest.py
[2020-07-21 16:09:35] INFO - prefect.TaskRunner | Task 'hello': Starting task run...
[2020-07-21 16:09:35] DEBUG - prefect.TaskRunner | Task 'hello': Handling state change from Pending to Running
[2020-07-21 16:09:35] DEBUG - prefect.TaskRunner | Task 'hello': Calling task.run() method...
hello
[2020-07-21 16:09:35] DEBUG - prefect.TaskRunner | Task 'hello': Handling state change from Running to Success
[2020-07-21 16:09:35] INFO - prefect.TaskRunner | Task 'hello': finished task run for task with final state: 'Success'
/opt/anaconda3/envs/prefect_dev/lib/python3.8/abc.py:102: size=26.8 KiB, count=241, average=114 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:203: size=11.9 KiB, count=23, average=531 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:141: size=11.2 KiB, count=102, average=113 B
/Users/laura/Development/prefect/src/prefect/engine/runner.py:48: size=4920 B, count=10, average=492 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:402: size=4608 B, count=9, average=512 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:143: size=4536 B, count=21, average=216 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:443: size=4296 B, count=75, average=57 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:434: size=2240 B, count=10, average=224 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/site-packages/box/box.py:178: size=1944 B, count=24, average=81 B
/opt/anaconda3/envs/prefect_dev/lib/python3.8/logging/__init__.py:664: size=1528 B, count=2, average=764 B
.
----------------------------------------------------------------------
Ran 1 test in 0.045s

OK
``` 

Resolving patch:
diff --git a/src/prefect/tasks/shell.py b/src/prefect/tasks/shell.py
--- a/src/prefect/tasks/shell.py
+++ b/src/prefect/tasks/shell.py
@@ -97,27 +97,29 @@ def run(self, command: str = None, env: dict = None) -> str:
                 tmp.write("\n".encode())
             tmp.write(command.encode())
             tmp.flush()
-            sub_process = Popen(
+            with Popen(
                 [self.shell, tmp.name], stdout=PIPE, stderr=STDOUT, env=current_env
-            )
-            lines = []
-            line = None
-            for raw_line in iter(sub_process.stdout.readline, b""):
-                line = raw_line.decode("utf-8").rstrip()
-                if self.return_all:
-                    lines.append(line)
-                else:
-                    # if we're returning all, we don't log every line
-                    self.logger.debug(line)
-            sub_process.wait()
-            if sub_process.returncode:
-                msg = "Command failed with exit code {}".format(sub_process.returncode,)
-                self.logger.error(msg)
+            ) as sub_process:
+                lines = []
+                line = None
+                for raw_line in iter(sub_process.stdout.readline, b""):
+                    line = raw_line.decode("utf-8").rstrip()
+                    if self.return_all:
+                        lines.append(line)
+                    else:
+                        # if we're returning all, we don't log every line
+                        self.logger.debug(line)
+                sub_process.wait()
+                if sub_process.returncode:
+                    msg = "Command failed with exit code {}".format(
+                        sub_process.returncode,
+                    )
+                    self.logger.error(msg)
 
-                if self.log_stderr:
-                    self.logger.error("\n".join(lines))
+                    if self.log_stderr:
+                        self.logger.error("\n".join(lines))
 
-                raise prefect.engine.signals.FAIL(msg) from None  # type: ignore
+                    raise prefect.engine.signals.FAIL(msg) from None  # type: ignore
         if self.return_all:
             return lines
         else:


Test patch: