Package: callisto-watchdog
This is a fork of https://github.com/gorakhargosh/watchdog with some specific tweaks for
[Callisto Jupyter D1 Servers](https://pypi.org/project/callisto-jupyter-d1/), you should
install [watchdog](https://pypi.org/project/watchdog/) instead if you’re looking for a stable APIPython API and shell utilities to monitor file system events.Works on 3.6+.If you want to use Python 2.6, you should stick with watchdog < 0.10.0.If you want to use Python 2.7, 3.4 or 3.5, you should stick with watchdog < 1.0.0.Callisto Watchdog DifferencesBaseObserver.scheduletakes anexclude_dirsparameter that is a list of directoriesthat watchdog will not descend into and observe, at the moment itonly works on Linux.
Any directory with one of those names in the directory tree will be ignored. Implementation is based on
<https://github.com/gorakhargosh/watchdog/issues/212#issuecomment-1078267199>This package gets published as callisto_watchdog to pypiExample API UsageA simple program that uses watchdog to monitor directories specified
as command-line arguments and logs events generated:importsysimporttimeimportloggingfromwatchdog.observersimportObserverfromwatchdog.eventsimportLoggingEventHandlerif__name__=="__main__":logging.basicConfig(level=logging.INFO,format='%(asctime)s-%(message)s',datefmt='%Y-%m-%d%H:%M:%S')path=sys.argv[1]iflen(sys.argv)>1else'.'event_handler=LoggingEventHandler()observer=Observer()observer.schedule(event_handler,path,recursive=True)observer.start()try:whileTrue:time.sleep(1)finally:observer.stop()observer.join()Shell UtilitiesWatchdog comes with anoptionalutility script calledwatchmedo.
Please typewatchmedo--helpat the shell prompt to
know more about this tool.Here is how you can log the current directory recursively
for events related only to*.pyand*.txtfiles while
ignoring all directory events:watchmedolog\--patterns="*.py;*.txt"\--ignore-directories\--recursive\.You can use theshell-commandsubcommand to execute shell commands in
response to events:watchmedoshell-command\--patterns="*.py;*.txt"\--recursive\--command='echo "${watch_src_path}"'\.Please see the help information for these commands by typing:watchmedo[command]--helpAboutwatchmedoTrickswatchmedocan readtricks.yamlfiles and execute tricks within them in
response to file system events. Tricks are actually event handlers that
subclasswatchdog.tricks.Trickand are written by plugin authors. Trick
classes are augmented with a few additional features that regular event handlers
don’t need.An exampletricks.yamlfile:tricks:-watchdog.tricks.LoggerTrick:patterns:["*.py","*.js"]-watchmedo_webtricks.GoogleClosureTrick:patterns:['*.js']hash_names:truemappings_format:json# json|yaml|pythonmappings_module:app/javascript_mappingssuffix:.min.jscompilation_level:advanced# simple|advancedsource_directory:app/static/js/destination_directory:app/public/js/files:index-page:-app/static/js/vendor/jquery*.js-app/static/js/base.js-app/static/js/index-page.jsabout-page:-app/static/js/vendor/jquery*.js-app/static/js/base.js-app/static/js/about-page/**/*.jsThe directory containing thetricks.yamlfile will be monitored. Each trick
class is initialized with its corresponding keys in thetricks.yamlfile as
arguments and events are fed to an instance of this class as they arrive.InstallationInstall from PyPI usingpip:$python-mpipinstall-Uwatchdog# or to install the watchmedo utility:$python-mpipinstall-U"watchdog[watchmedo]"Install from source:$python-mpipinstall-e.# or to install the watchmedo utility:$python-mpipinstall-e".[watchmedo]"Installation CaveatsThewatchmedoscript depends onPyYAMLwhich links withLibYAML,
which brings a performance boost to the PyYAML parser. However, installingLibYAMLis optional but recommended. On macOS, you can usehomebrewto install LibYAML:$brewinstalllibyamlOn Linux, use your favorite package manager to install LibYAML. Here’s how you
do it on Ubuntu:$sudoaptinstalllibyaml-devOn Windows, please installPyYAMLusing the binaries they provide.DocumentationYou can browse the latest releasedocumentationonline.ContributeFork therepositoryon GitHub and send a pull request, or file an issue
ticket at theissue tracker. For general help and questions use the officialmailing listor ask onstackoverflowwith tagpython-watchdog.Create and activate your virtual environment, then:python -m pip install pytest pytest-cov
python -m pip install -e ".[watchmedo]"
python -m pytest testsIf you are making a substantial change, add an entry to the “Unreleased” section
of thechangelog.Supported PlatformsLinux 2.6 (inotify)macOS (FSEvents, kqueue)FreeBSD/BSD (kqueue)Windows (ReadDirectoryChangesW with I/O completion ports;
ReadDirectoryChangesW worker threads)OS-independent (polling the disk for directory snapshots and comparing them
periodically; slow and not recommended)Note that when using watchdog with kqueue, you need the
number of file descriptors allowed to be opened by programs
running on your system to be increased to more than the
number of files that you will be monitoring. The easiest way
to do that is to edit your~/.profilefile and add
a line similar to:ulimit -n 1024This is an inherent problem with kqueue because it uses
file descriptors to monitor files. That plus the enormous
amount of bookkeeping that watchdog needs to do in order
to monitor file descriptors just makes this a painful way
to monitor files and directories. In essence, kqueue is
not a very scalable way to monitor a deeply nested
directory of files and directories with a large number of
files.About using watchdog with editors like VimVim does not modify files unless directed to do so.
It creates backup files and then swaps them in to replace
the files you are editing on the disk. This means that
if you use Vim to edit your files, the on-modified events
for those files will not be triggered by watchdog.
You may need to configure Vim appropriately to disable
this feature.About using watchdog with CIFSWhen you want to watch changes in CIFS, you need to explicitly tell watchdog to
usePollingObserver, that is, instead of letting watchdog decide an
appropriate observer like in the example above, do:from watchdog.observers.polling import PollingObserver as ObserverDependenciesPython 3.6 or above.XCode(only on macOS when installing from sources)PyYAML(only forwatchmedo)LicensingWatchdog is licensed under the terms of theApache License, version 2.0.Copyright 2011Yesudeep Mangalapilly.Copyright 2012 Google, Inc & contributors.Projectsource codeis available at Github. Please report bugs and file
enhancement requests at theissue tracker.Why Watchdog?Too many people tried to do the same thing and none did what I needed Python
to do:pnotifyunison fsmonitorfsmonitorguardpyinotifyinotify-toolsjnotifytreewatcherfile.monitorpyfilesystemChangelog2.1.102022-xx-xx •full history[documentation] HTML documentation builds are now tested for errors.Thanks to our beloved contributors: @kurtmckee2.1.92022-06-10 •full history[fsevents] Fix flakey test to assert that there are no errors when stopping the emitter.[inotify] Suppress occasionalOSError: [Errno 9] Bad file descriptorat shutdown. (#805)[watchmedo] Makeauto-restartrestart the sub-process if it terminates. (#896)[watchmedo] Avoid zombie sub-processes when runningshell-commandwithout--wait. (#405)Thanks to our beloved contributors: @samschott, @taleinat, @altendky, @BoboTiG2.1.82022-05-15 •full historyFix adding failed emitters on observer schedule. (#872)[inotify] Fix hang when unscheduling watch on a path in an unmounted filesystem. (#869)[watchmedo] Fix broken parsing of--kill-afterargument for theauto-restartcommand. (#870)[watchmedo] Fix broken parsing of boolean arguments. (#887)[watchmedo] Fix broken parsing of commands fromauto-restart, andshell-command. (#888)[watchmedo] Support setting verbosity level via-q/--quietand-v/--verbosearguments. (#889)Thanks to our beloved contributors: @taleinat, @kianmeng, @palfrey, @IlayRosenberg, @BoboTiG2.1.72022-03-25 •full historyEliminate timeout in waiting on event queue. (#861)[inotify] Fixnotequality implementation forInotifyEvent. (#848)[watchmedo] Fix calling commands from within a Python script. (#879)[watchmedo]PyYAMLis loaded only when strictly necessary. Simple usages ofwatchmedoare possible without the module being installed. (#847)Thanks to our beloved contributors: @sattlerc, @JanzenLiu, @BoboTiG2.1.62021-10-01 •full history[bsd] Fixed returned paths inkqueue.pyand restored the overall results of the test suite. (#842)[bsd] Updated FreeBSD CI support .(#841)[watchmedo] Removed thearghdependency in favor of the builtinargparsemodule. (#836)[watchmedo] Removed unexistantWindowsApiAsyncObserverreferences and--debug-force-winapi-asyncarguments.[watchmedo] Improved the help output.Thanks to our beloved contributors: @knobix, @AndreaRe9, @BoboTiG2.1.52021-08-23 •full historyFix regression introduced in 2.1.4 (reverted “Allow overriding or adding custom event handlers to event dispatch map. (#814)”). (#830)Convert regexes of typestrtolist. (831)Thanks to our beloved contributors: @unique1o1, @BoboTiG2.1.42021-08-19 •full history[watchmedo] Fix usage ofos.setsid()andos.killpg()Unix-only functions. (#809)[mac] Fix missingFileModifiedEventon permission or ownership changes of a file. (#815)[mac] Convert absolute watch path inFSEeventsEmitterwithos.path.realpath(). (#822)Fix a possibleAttributeErrorinSkipRepeatsQueue._put(). (#818)Allow overriding or adding custom event handlers to event dispatch map. (#814)Fix tests on big endian platforms. (#828)Thanks to our beloved contributors: @replabrobin, @BoboTiG, @SamSchott, @AndreiB97, @NiklasRosenstein, @ikokollari, @mgorny2.1.32021-06-26 •full historyPublish macOSarm64anduniversal2wheels. (#740)Thanks to our beloved contributors: @kainjow, @BoboTiG2.1.22021-05-19 •full history[mac] Fix relative path handling for non-recursive watch. (#797)[windows] On PyPy, events happening right afterstart()were missed. Add a workaround for that. (#796)Thanks to our beloved contributors: @oprypin, @CCP-Aporia, @BoboTiG2.1.12021-05-10 •full history[mac] Fix callback exceptions when the watcher is deleted but still receiving events (#786)Thanks to our beloved contributors: @rom1win, @BoboTiG, @CCP-Aporia2.1.02021-05-04 •full history[inotify] Simplifylibcloading (#776)[mac] Add support for non-recursive watches inFSEventsEmitter(#779)[watchmedo] Add support for--debug-force-*arguments totricks(#781)Thanks to our beloved contributors: @CCP-Aporia, @aodj, @UnitedMarsupials, @BoboTiG2.0.32021-04-22 •full history[mac] Uselogger.debug()instead oflogger.info()(#774)Updated documentation links (#777)Thanks to our beloved contributors: @globau, @imba-tjd, @BoboTiG2.0.22021-02-22 •full history[mac] Add missing exception objects (#766)Thanks to our beloved contributors: @CCP-Aporia, @BoboTiG2.0.12021-02-17 •full history[mac] Fix a segmentation fault when dealing with unicode paths (#763)Moved the CI from Travis-CI to GitHub Actions (#764)Thanks to our beloved contributors: @SamSchott, @BoboTiG2.0.02021-02-11 •full historyAvoid deprecatedPyEval_InitThreadson Python 3.7+ (#746)[inotify] Add support forIN_CLOSE_WRITEevents. AFileCloseEventevent will be fired. Note thatIN_CLOSE_NOWRITEevents are not handled to prevent much noise. (#184,#245,#280,#313,#690)[inotify] Allow to stop the emitter multiple times (#760)[mac] Support coalesced filesystem events (#734)[mac] Drop support for macOS 10.12 and earlier (#750)[mac] Fix an issue when renaming an item changes only the casing (#750)Thanks to our beloved contributors: @bstaletic, @lukassup, @ysard, @SamSchott, @CCP-Aporia, @BoboTiG1.0.22020-12-18 •full historyWheels are published for GNU/Linux, macOS and Windows (#739)[mac] Fix missingevent_idattribute infsevents(#721)[mac] Return byte paths if a byte path was given infsevents(#726)[mac] Add compatibility with old macOS versions (#733)Uniformize event for deletion of watched dir (#727)Thanks to our beloved contributors: @SamSchott, @CCP-Aporia, @di, @BoboTiG1.0.12020-12-10 • Fix version with good metadatas.1.0.02020-12-10 •full historyVersioning is now following thesemverDrop support for Python 2.7, 3.4 and 3.5[mac] Regression fixes for nativefsevents(#717)[windows]winapi.BUFFER_SIZEnow defaults to64000(instead of2048) (#700)[windows] Introducedwinapi.PATH_BUFFER_SIZE(defaults to2048) to keep the old behavior with path-realted functions (#700)Usepathlibfrom the standard library, instead of pathtools (#556)Allow file paths on Unix that don’t follow the file system encoding (#703)Removed the long-time deprecatedevents.LoggingFileSystemEventHandlerclass, useLoggingEventHandlerinsteadThanks to our beloved contributors: @SamSchott, @bstaletic, @BoboTiG, @CCP-Aporia0.10.42020-11-21 •full historyAddloggerparameter for theLoggingEventHandler(#676)Replace mutable default arguments withif Noneimplementation (#677)Expand tests to Python 2.7 and 3.5-3.10 for GNU/Linux, macOS and Windows[mac] Performance improvements for thefseventsmodule (#680)[mac] Prevent compilation ofwatchdog_fsevents.con non-macOS machines (#687)[watchmedo] Handle shutdown events fromSIGTERMandSIGINTmore reliably (#693)Thanks to our beloved contributors: @Sraw, @CCP-Aporia, @BoboTiG, @maybe-sybr0.10.32020-06-25 •full historyEnsureObservedWatch.pathis a string (#651)[inotify] Allow to monitor single file (#655)[inotify] Prevent raising an exception when a file in a monitored folder has no permissions (#669,#670)Thanks to our beloved contributors: @brant-ruan, @rec, @andfoy, @BoboTiG0.10.22020-02-08 •full historyFixed thebuild_extcommand on macOS Catalina (#628)Fixed the installation of macOS requirements on non-macOS OSes (#635)Refactoreddispatch()method ofFileSystemEventHandler,PatternMatchingEventHandlerandRegexMatchingEventHandler[bsd] Improved tests support on non Windows/Linux platforms (#633,#639)[bsd] Added FreeBSD CI support (#532)[bsd] Restored full support (#638,#641)Thanks to our beloved contributors: @BoboTiG, @evilham, @danilobellini0.10.12020-01-30 •full historyFixed Python 2.7 to 3.6 installation when the OS locale is set to POSIX (#615)Fixed thebuild_extcommand on macOS  (#618,#620)Moved requirements tosetup.cfg(#617)[mac] Removed old C code for Python 2.5 in thefseventsC implementation[snapshot] AddedEmptyDirectorySnapshot(#613)Thanks to our beloved contributors: @Ajordat, @tehkirill, @BoboTiG0.10.02020-01-26 •full historyBreaking ChangesDropped support for Python 2.6, 3.2 and 3.3Emitters that failed to start are now removed[snapshot] Removed the deprecatedwalker_callbackargument,
usestatinstead[watchmedo] The utility is no more installed by default but via the extrawatchdog[watchmedo]Other ChangesFixed several Python 3 warningsIdentify synthesized events withis_syntheticattribute (#369)Useos.scandir()to improve memory usage (#503)[bsd] Fixed flavors of FreeBSD detection (#529)[bsd] Skip unprocessable socket files (#509)[inotify] Fixed events containing non-ASCII characters (#516)[inotify] Fixed the wayOSErrorare re-raised (#377)[inotify] Fixed wrong source path after renaming a top level folder (#515)[inotify] Removed  delay from non-move events (#477)[mac] Fixed a bug when callingFSEventsEmitter.stop()twice (#466)[mac] Support for unscheduling deleted watch (#541)[mac] Fixed missing field initializers and unused parameters inwatchdog_fsevents.c[snapshot] Don’t walk directories without read permissions (#408)[snapshot] Fixed a race condition crash when a directory is swapped for a file (#513)[snasphot] Fixed anAttributeErrorabout forgottenpath_for_inodeattr (#436)[snasphot] Added theignore_device=Falseparameter to the ctor (597)[watchmedo] Fixed the path separator used (#478)[watchmedo] Fixed the use ofyaml.load()foryaml.safe_load()(#453)[watchmedo] Handle all available signals (#549)[watchmedo] Added the--debug-force-pollingargument (#404)[windows] Fixed issues when the observed directory is deleted (#570and#601)[windows]WindowsApiEmittermade easier to subclass (#344)[windows] Use separate ctypes DLL instances[windows] Generate sub created events only ifrecursive=True(#454)Thanks to our beloved contributors: @BoboTiG, @LKleinNux, @rrzaripov,
@wildmichael, @TauPan, @segevfiner, @petrblahos, @QuantumEnergyE,
@jeffwidman, @kapsh, @nickoala, @petrblahos, @julianolf, @tonybaloney,
@mbakiev, @pR0Ps, javaguirre, @skurfer, @exarkun, @joshuaskelly,
@danilobellini, @Ajordat0.9.02018-08-28 •full historyDeleting the observed directory now emits aDirDeletedEventevent[bsd] Improved the platform detection (#378)[inotify] Fixed a crash when the root directory being watched by was deleted (#374)[inotify] Handle systems providing uClibc[linux] Fixed a possibleDirDeletedEventduplication when
deleting a directory[mac] Fixed unicode path handlingfsevents2.py(#298)[watchmedo] Added the--debug-force-pollingargument (#336)[windows] Fixed theFILE_LIST_DIRECTORYconstant (#376)Thanks to our beloved contributors: @vulpeszerda, @hpk42, @tamland, @senden9,
@gorakhargosh, @nolsto, @mafrosis, @DonyorM, @anthrotype, @danilobellini,
@pierregr, @ShinNoNoir, @adrpar, @gforcada, @pR0Ps, @yegorich, @dhke0.8.32015-02-11 •full historyFixed the use of the root logger (#274)[inotify] Refactored libc loading and improved error handling ininotify_c.py[inotify] Fixed a possible unbound local error ininotify_c.pyThanks to our beloved contributors: @mmorearty, @tamland, @tony,
@gorakhargosh0.8.22014-10-29 •full historyEvent emitters are no longer started on schedule ifObserveris not
already running[mac] Fixed usued arguments to pass clang compilation (#265)[snapshot] Fixed a possible race condition crash on directory deletion (#281)[windows] Fixed an error when watching the same folder again (#270)Thanks to our beloved contributors: @tamland, @apetrone, @Falldog,
@theospears0.8.12014-07-28 •full historyFixedanon_inodedescriptors leakage  (#249)[inotify] Fixed thread stop dead lock (#250)Thanks to our beloved contributors: @Witos, @adiroiban, @tamland0.8.02014-07-02 •full historyFixedarghdeprecation warnings (#242)[snapshot] Methods returning internal stats info were replaced bymtime(),inode()andpath()methods[snapshot] Deprecated thewalker_callbackargument[watchmedo] Fixedauto-restartto terminate all children processes (#225)[watchmedo] Added the--no-parallelargument (#227)[windows] Fixed the value ofINVALID_HANDLE_VALUE(#123)[windows] Fixed octal usages to work with Python 3 as well (#223)Thanks to our beloved contributors: @tamland, @Ormod, @berdario, @cro,
@BernieSumption, @pypingou, @gotcha, @tommorris, @frewsxcv