Repository: prowler-cloud/prowler
Pull request: fix(ulimit): handle low ulimit OSError
### Description

Handle low ulimit OSError by giving more information in the error and giving a warning to the user.

### License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.


Problem statement:
OSError[13] handling enhancement (ulimit -n 4096) toubleshooting advice
**Is your feature request related to a problem? Please describe.**
A colleague of mine ran into the `OSError[13] error`. I could help her debug it by increasing the `ulimit` but we could add a check to prowler that warns the user at the start that if they are running in linux and ulimit  -n isn't set to at least 4096 (I ran into issues running it with 1000, which is what the troubleshooting page recommends) it may result in an `OSError[13] error`

**Describe the solution you'd like**
Add a check to prowler that warns the user at the start that if they are running in linux and ulimit  -n isn't set to at least 4096 (I ran into issues running it with 1000, which is what the troubleshooting page recommends) it may result in an `OSError[13] error`

**Describe alternatives you've considered**
Prowler could also link to the troubleshooting page if an error is encountered, and ask the user to provide feedback if the troubleshooting page does not assist them.

**Additional context**
None

Hints from issue discussion:

Commit message: handle low ulimit

Patch to review:
diff --git a/prowler/lib/check/check.py b/prowler/lib/check/check.py
index 367a6ecd87..69ed5c9905 100644
--- a/prowler/lib/check/check.py
+++ b/prowler/lib/check/check.py
@@ -4,6 +4,7 @@
 import sys
 import traceback
 from pkgutil import walk_packages
+from resource import RLIMIT_NOFILE, getrlimit
 from types import ModuleType
 from typing import Any
 
@@ -110,6 +111,7 @@ def parse_checks_from_file(input_file: str, provider: str) -> set:
     checks_to_execute = set()
     f = open_file(input_file)
     json_file = parse_json_file(f)
+    f.close()
 
     for check_name in json_file[provider]:
         checks_to_execute.add(check_name)
@@ -356,6 +358,13 @@ def execute_checks(
         audit_progress=0,
     )
 
+    # Check ulimit for the maximum system open files
+    soft, _ = getrlimit(RLIMIT_NOFILE)
+    if soft < 4096:
+        logger.warning(
+            f"Prowler may have errors because of your user session maximum open files ({soft}). To solve this issue, increase the shell session limit by running this command `ulimit -n 4096`. More info in https://docs.prowler.cloud/en/latest/troubleshooting/"
+        )
+
     # Execution with the --only-logs flag
     if audit_output_options.only_logs:
         for check_name in checks_to_execute:
diff --git a/prowler/lib/utils/utils.py b/prowler/lib/utils/utils.py
index 3d9137faed..77ea567807 100644
--- a/prowler/lib/utils/utils.py
+++ b/prowler/lib/utils/utils.py
@@ -17,9 +17,14 @@ def open_file(input_file: str, mode: str = "r") -> TextIOWrapper:
     try:
         f = open(input_file, mode)
     except Exception as e:
-        logger.critical(
-            f"{input_file}: {e.__class__.__name__}[{e.__traceback__.tb_lineno}]"
-        )
+        if e.__class__.__name__ == "OSError":
+            logger.critical(
+                "Ooops! You reached your user session maximum open files. To solve this issue, increase the shell session limit by running this command `ulimit -n 4096`. More info in https://docs.prowler.cloud/en/latest/troubleshooting/"
+            )
+        else:
+            logger.critical(
+                f"{input_file}: {e.__class__.__name__}[{e.__traceback__.tb_lineno}]"
+            )
         sys.exit(1)
     else:
         return f
diff --git a/prowler/providers/aws/aws_provider.py b/prowler/providers/aws/aws_provider.py
index 11126a98f5..07d678bbd4 100644
--- a/prowler/providers/aws/aws_provider.py
+++ b/prowler/providers/aws/aws_provider.py
@@ -112,6 +112,7 @@ def generate_regional_clients(
         actual_directory = pathlib.Path(os.path.dirname(os.path.realpath(__file__)))
         f = open_file(f"{actual_directory}/{aws_services_json_file}")
         data = parse_json_file(f)
+        f.close()
         # Check if it is a subservice
         json_regions = data["services"][service]["regions"][
             audit_info.audited_partition
@@ -146,6 +147,7 @@ def get_aws_available_regions():
         actual_directory = pathlib.Path(os.path.dirname(os.path.realpath(__file__)))
         f = open_file(f"{actual_directory}/{aws_services_json_file}")
         data = parse_json_file(f)
+        f.close()
 
         regions = set()
         for service in data["services"].values():


Code review comments:
diff_hunk: @@ -110,6 +111,7 @@ def parse_checks_from_file(input_file: str, provider: str) -> set:
     checks_to_execute = set()
     f = open_file(input_file)
     json_file = parse_json_file(f)
+    f.close()
path: prowler/lib/check/check.py
text: @user1:
I prefer if you use the following with the context manager:
```python
with open("hello.txt") as my_file:
    print(my_file.read())
```

@author:
Done

diff_hunk: @@ -17,9 +17,14 @@ def open_file(input_file: str, mode: str = "r") -> TextIOWrapper:
     try:
         f = open(input_file, mode)
     except Exception as e:
-        logger.critical(
-            f"{input_file}: {e.__class__.__name__}[{e.__traceback__.tb_lineno}]"
-        )
+        if e.__class__.__name__ == "OSError":
path: prowler/lib/utils/utils.py
text: @user1:
Why don't you do `except OSError as err:`?

I think is clearer and then you can handle other exceptions

@author:
Sure!

diff_hunk: @@ -17,9 +17,14 @@ def open_file(input_file: str, mode: str = "r") -> TextIOWrapper:
     try:
         f = open(input_file, mode)
     except Exception as e:
-        logger.critical(
-            f"{input_file}: {e.__class__.__name__}[{e.__traceback__.tb_lineno}]"
-        )
+        if e.__class__.__name__ == "OSError":
+            logger.critical(
+                "Ooops! You reached your user session maximum open files. To solve this issue, increase the shell session limit by running this command `ulimit -n 4096`. More info in https://docs.prowler.cloud/en/latest/troubleshooting/"
path: prowler/lib/utils/utils.py
text: @user1:
```suggestion
                "Ooops! You reached your user session maximum open files. To solve this issue, increase the shell session limit by running this command `ulimit -n 4096`. For more info visit https://docs.prowler.cloud/en/latest/troubleshooting/"
```

@author:
Done

diff_hunk: @@ -356,6 +358,13 @@ def execute_checks(
         audit_progress=0,
     )
 
+    # Check ulimit for the maximum system open files
+    soft, _ = getrlimit(RLIMIT_NOFILE)
+    if soft < 4096:
+        logger.warning(
+            f"Prowler may have errors because of your user session maximum open files ({soft}). To solve this issue, increase the shell session limit by running this command `ulimit -n 4096`. More info in https://docs.prowler.cloud/en/latest/troubleshooting/"
path: prowler/lib/check/check.py
text: @user1:
```suggestion
            f"Your session file descriptors limit ({soft} open files) is below 4096. We recommend to increase it to avoid errors. Solve it running this command `ulimit -n 4096`. For more info visit https://docs.prowler.cloud/en/latest/troubleshooting/"
```