Repository: spyder-ide/spyder
Pull request: PR: Handle inotify error on Linux
<!--- Make sure to read the Contributing Guidelines:                   --->
<!--- https://github.com/spyder-ide/spyder/blob/master/CONTRIBUTING.md --->
<!--- and follow PEP 8, PEP 257 and Spyder's code style:               --->
<!--- https://github.com/spyder-ide/spyder/wiki/Dev:-Coding-Style      --->

## Description of Changes


* [ ] Wrote at least one-line docstrings (for any new functions)
* [ ] Added unit test(s) covering the changes (if testable)
<!--- Remember that an image/animation is worth a thousand words! --->
* [x] Included a screenshot or animation (if affecting the UI, see [Licecap](https://www.cockos.com/licecap/))


<!--- Explain what you've done and why --->

![image](https://user-images.githubusercontent.com/16781833/67305959-d1f3fa80-f4bb-11e9-9507-d2ebc010fd2c.png)





### Issue(s) Resolved

<!--- List the issue(s) below, in the form "Fixes #1234"; one per line --->

Fixes #10478 


### Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the [Developer Certificate of Origin](https://developercertificate.org)
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

<!--- TYPE YOUR USER/NAME AFTER THE FOLLOWING: --->
I certify the above statement is true and correct: dalthviz

<!--- Thanks for your help making Spyder better for everyone! --->


Problem statement:
"OSError: inotify watch limit reached" with many files in a project
spyder4b7 gives this error if there are a lot of files in the project (150000). I use it for machine learning and it is not unusual to have a subdirectory with that number of files in a project. 

`sudo sysctl fs.inotify.max_user_watches=500000 `

makes it work on ubuntu 18.04, but this is not really nice and there is no real help from the crash report:
```
#spyder3
Attribute Qt::AA_ShareOpenGLContexts must be set before QCoreApplication is created.
Attribute Qt::AA_UseSoftwareOpenGL must be set before QCoreApplication is created.
Traceback (most recent call last):
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/spyder/app/mainwindow.py", line 3646, in main
    mainwindow = run_spyder(app, options, args)
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/spyder/app/mainwindow.py", line 3498, in run_spyder
    main.post_visible_setup()
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/spyder/app/mainwindow.py", line 1415, in post_visible_setup
    self.projects.reopen_last_project()
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/spyder/plugins/projects/plugin.py", line 404, in reopen_last_project
    save_previous_files=False)
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/spyder/plugins/projects/plugin.py", line 322, in open_project
    self.watcher.start(path)
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/spyder/plugins/projects/utils/watcher.py", line 93, in start
    self.observer.start()
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/observers/api.py", line 255, in start
    emitter.start()
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/utils/__init__.py", line 110, in start
    self.on_thread_start()
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/observers/inotify.py", line 121, in on_thread_start
    self._inotify = InotifyBuffer(path, self.watch.is_recursive)
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/observers/inotify_buffer.py", line 35, in __init__
    self._inotify = Inotify(path, recursive)
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 199, in __init__
    self._add_dir_watch(path, recursive, event_mask)
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 387, in _add_dir_watch
    self._add_watch(full_path, mask)
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 401, in _add_watch
    Inotify._raise_error()
  File "/home/detlef/tf-2-nightly-gpu/lib/python3.6/site-packages/watchdog/observers/inotify_c.py", line 413, in _raise_error
    raise OSError("inotify watch limit reached")
OSError: inotify watch limit reached
 
```

Hints from issue discussion:
@dalthviz, please take a look at this one.

The idea is to catch this specific error and explain users how to increase the inotify limit on Linux in a QMessageBox (VSCode does the same, by the way).

Commit message: Projects: Add handling for inotify watch limit reached

Patch to review:
diff --git a/spyder/plugins/projects/utils/watcher.py b/spyder/plugins/projects/utils/watcher.py
index 33e2e2aabd0..d8a17fd67d6 100644
--- a/spyder/plugins/projects/utils/watcher.py
+++ b/spyder/plugins/projects/utils/watcher.py
@@ -11,10 +11,14 @@
 
 # Third-party imports
 from qtpy.QtCore import QObject, Signal
+from qtpy.QtWidgets import QMessageBox
 
 from watchdog.observers import Observer
 from watchdog.events import FileSystemEventHandler
 
+# Local imports
+from spyder.config.base import _
+
 logger = logging.getLogger(__name__)
 
 
@@ -87,10 +91,28 @@ def connect_signals(self, project):
         self.event_handler.sig_file_modified.connect(project.file_modified)
 
     def start(self, workspace_folder):
-        self.observer = Observer()
-        self.observer.schedule(
-            self.event_handler, workspace_folder, recursive=True)
-        self.observer.start()
+        # Needed to handle an error caused for inotify watch limit reached.
+        # See spyder-ide/spyder#10478
+        try:
+            self.observer = Observer()
+            self.observer.schedule(
+                self.event_handler, workspace_folder, recursive=True)
+            self.observer.start()
+        except OSError:
+            QMessageBox.warning(self.parent(), "Spyder",
+                                _("An error occurred while creating "
+                                  "the watcher of changes for the working "
+                                  "directory. Please, try to increase "
+                                  "the inotify limit on your system.<br><br>"
+                                  "On Linux you can use the following "
+                                  "command:<br><br>"
+                                  "<code>sudo sysctl -n -w "
+                                  "fs.inotify.max_user_watches=524288</code>"
+                                  "<br><br>For a permanent setting of the "
+                                  "configuration you will need to edit "
+                                  "<code>/etc/sysctl.conf</code> an add "
+                                  "the following line:<br><br> <code>"
+                                  "fs.inotify.max_user_watches=524288</code>"))
 
     def stop(self):
         self.observer.stop()


Code review comments:
diff_hunk: @@ -87,10 +91,28 @@ def connect_signals(self, project):
         self.event_handler.sig_file_modified.connect(project.file_modified)
 
     def start(self, workspace_folder):
-        self.observer = Observer()
-        self.observer.schedule(
-            self.event_handler, workspace_folder, recursive=True)
-        self.observer.start()
+        # Needed to handle an error caused for inotify watch limit reached.
+        # See spyder-ide/spyder#10478
+        try:
+            self.observer = Observer()
+            self.observer.schedule(
+                self.event_handler, workspace_folder, recursive=True)
+            self.observer.start()
+        except OSError:
+            QMessageBox.warning(self.parent(), "Spyder",
+                                _("An error occurred while creating "
+                                  "the watcher of changes for the working "
+                                  "directory. Please, try to increase "
+                                  "the inotify limit on your system.<br><br>"
+                                  "On Linux you can use the following "
+                                  "command:<br><br>"
+                                  "<code>sudo sysctl -n -w "
+                                  "fs.inotify.max_user_watches=524288</code>"
+                                  "<br><br>For a permanent setting of the "
+                                  "configuration you will need to edit "
+                                  "<code>/etc/sysctl.conf</code> an add "
+                                  "the following line:<br><br> <code>"
+                                  "fs.inotify.max_user_watches=524288</code>"))
path: spyder/plugins/projects/utils/watcher.py
text: @user1:
Let's change this message to be:

File changes for this project can't be tracked because it contains too many files. To fix this you need to increase the inotify limit in your system, with the following command:

    sudo sysctl -n -w fs.inotify.max_user_watches=524288

For a permanent solution you need to add to `/etc/sysctl.conf` the following line

    fs.inotify.max_user_watches=524288

After doing that, you need to close and start Spyder again so those changes can take effect.

@user1:
Also, please indent things like this to have more space to write the message:

```python
QMessageBox.warning(
    self.parent(),
    "Project warning",
    "File changes ..."
)

@user1:
And finally, please make `self.observer = None` above the `QMessageBox` because we won't be able to use it for this project.

diff_hunk: @@ -87,10 +91,28 @@ def connect_signals(self, project):
         self.event_handler.sig_file_modified.connect(project.file_modified)
 
     def start(self, workspace_folder):
-        self.observer = Observer()
-        self.observer.schedule(
-            self.event_handler, workspace_folder, recursive=True)
-        self.observer.start()
+        # Needed to handle an error caused for inotify watch limit reached.
+        # See spyder-ide/spyder#10478
+        try:
+            self.observer = Observer()
+            self.observer.schedule(
+                self.event_handler, workspace_folder, recursive=True)
+            self.observer.start()
+        except OSError:
path: spyder/plugins/projects/utils/watcher.py
text: @user1:
Let's capture the error here and verify that `inotify` is present in the message error message, like this

```python
except OSError as e:
    if u`inotify` in e:
        QMessageBox.warning(...)
```

Besides, if `inotify` is not part of the message, we need to raise the error generated at this point.

diff_hunk: @@ -87,10 +91,28 @@ def connect_signals(self, project):
         self.event_handler.sig_file_modified.connect(project.file_modified)
 
     def start(self, workspace_folder):
-        self.observer = Observer()
-        self.observer.schedule(
-            self.event_handler, workspace_folder, recursive=True)
-        self.observer.start()
+        # Needed to handle an error caused for inotify watch limit reached.
path: spyder/plugins/projects/utils/watcher.py
text: @user1:
Let's change this to

    # Needed to handle an error caused for inotify limit reached.