Repository: gorakhargosh/watchdog

Issue:
Can't pickle DirectorySnapshots
First of all, I'm not sure if this is an issue, an expected behaviour or just an scenario not aimed to be solved.

## Expected Behavior
I expect to be able to pickle DirectorySnapshot objects, as I can do in the latest release, 0.9.0.

## Current Behavior
I can't pickle a DirectorySnapshot because I'm getting a PicklingError.
It doesn't happen on the 0.9.0 version.

## Possible Solution
One possible workaround is using the `dill` library and it would work, but I find it strange to not make it work with `pickle` like it did before because it breaks existing code.

The following works:
```python
import dill
from src.watchdog.utils.dirsnapshot import DirectorySnapshot

dill.dumps(DirectorySnapshot("/Users/alex/tmp"))
```

## Steps to Reproduce
1. Clone the repository with the latest `master` commit. Currently `7a55f317946953d86a5acea64d9b31931ca25307`.
2. Try to pickle a DirectorySnapshot. Notice the import from `src` directory, not the pypi version.
```python
import pickle
from src.watchdog.utils.dirsnapshot import DirectorySnapshot

pickle.dumps(DirectorySnapshot("/Users/alex/tmp"))
```
3. You will see the following exception:
```python
_pickle.PicklingError: Can't pickle <function DirectorySnapshot.<lambda> at 0x10dfb20d0>: attribute lookup DirectorySnapshot.<lambda> on src.watchdog.utils.dirsnapshot failed
```

## Context (Environment)
I'm facing this problem while using both python 3.7 and python 3.8 in a macOS Mojave 10.14.6.

In my scenario I need to pickle a DirectorySnapshot object to later compare it with the current snapshot and see if there was any change while my program was down.

## Possible Implementation
I just propose to make it work with `pickle`. Maybe the problematic lambda is not required and we can just avoid serializing it using `__getstate__`.

Otherwise it just doesn't have a solution and we have to change to using the `dill` library.



Comments:

Patch:
diff --git a/changelog.rst b/changelog.rst
index e212e70..4cb688a 100644
--- a/changelog.rst
+++ b/changelog.rst
@@ -13,6 +13,8 @@ Breaking Changes
 
 - Dropped support for Python 2.6, 3.2 and 3.3
 - Emitters that failed to start are now removed
+- [snapshot] Removed the deprecated ``walker_callback`` argument,
+  use ``stat`` instead
 - [watchmedo] The utility is no more installed by default but via the extra
   ``watchdog[watchmedo]``
 
diff --git a/src/watchdog/observers/kqueue.py b/src/watchdog/observers/kqueue.py
index ad54911..0e4e5bb 100644
--- a/src/watchdog/observers/kqueue.py
+++ b/src/watchdog/observers/kqueue.py
@@ -435,12 +435,14 @@ class KqueueEmitter(EventEmitter):
         # A collection of KeventDescriptor.
         self._descriptors = KeventDescriptorSet()
 
-        def walker_callback(path, stat_info, self=self):
+        def custom_stat(path, self=self):
+            stat_info = stat(path)
             self._register_kevent(path, stat.S_ISDIR(stat_info.st_mode))
+            return stat_info
 
         self._snapshot = DirectorySnapshot(watch.path,
-                                           watch.is_recursive,
-                                           walker_callback)
+                                           recursive=watch.is_recursive,
+                                           stat=custom_stat)
 
     def _register_kevent(self, path, is_directory):
         """
diff --git a/src/watchdog/utils/dirsnapshot.py b/src/watchdog/utils/dirsnapshot.py
index 40ff95b..4083b3f 100644
--- a/src/watchdog/utils/dirsnapshot.py
+++ b/src/watchdog/utils/dirsnapshot.py
@@ -226,31 +226,27 @@ class DirectorySnapshot(object):
         snapshot; ``False`` otherwise.
     :type recursive:
         ``bool``
-    :param walker_callback:
-        .. deprecated:: 0.7.2
     :param stat:
         Use custom stat function that returns a stat structure for path.
         Currently only st_dev, st_ino, st_mode and st_mtime are needed.
 
-        A function with the signature ``walker_callback(path, stat_info)``
-        which will be called for every entry in the directory tree.
+        A function taking a ``path`` as argument which will be called
+        for every entry in the directory tree.
     :param listdir:
         Use custom listdir function. For details see ``os.scandir`` if available, else ``os.listdir``.
     """
 
     def __init__(self, path, recursive=True,
-                 walker_callback=(lambda p, s: None),
                  stat=default_stat,
                  listdir=scandir):
         self.recursive = recursive
-        self.walker_callback = walker_callback
         self.stat = stat
         self.listdir = listdir
 
         self._stat_info = {}
         self._inode_to_path = {}
 
-        st = stat(path)
+        st = self.stat(path)
         self._stat_info[path] = st
         self._inode_to_path[(st.st_ino, st.st_dev)] = path
 
@@ -258,7 +254,6 @@ class DirectorySnapshot(object):
             i = (st.st_ino, st.st_dev)
             self._inode_to_path[i] = p
             self._stat_info[p] = st
-            walker_callback(p, st)
 
     def walk(self, root):
         try:


Test patch:
diff --git a/tests/test_snapshot_diff.py b/tests/test_snapshot_diff.py
index 9cae27d..477f041 100644
--- a/tests/test_snapshot_diff.py
+++ b/tests/test_snapshot_diff.py
@@ -16,12 +16,15 @@
 
 import errno
 import os
+import pickle
 import time
-from .shell import mkdir, touch, mv, rm
+
 from watchdog.utils.dirsnapshot import DirectorySnapshot
 from watchdog.utils.dirsnapshot import DirectorySnapshotDiff
 from watchdog.utils import platform
 
+from .shell import mkdir, touch, mv, rm
+
 
 def wait():
     """
@@ -35,6 +38,13 @@ def wait():
         time.sleep(0.5)
 
 
+def test_pickle(p):
+    """It should be possible to pickle a snapshot."""
+    mkdir(p('dir1'))
+    snasphot = DirectorySnapshot(p('dir1'))
+    pickle.dumps(snasphot)
+
+
 def test_move_to(p):
     mkdir(p('dir1'))
     mkdir(p('dir2'))