WATCHDOG(3) watchdog WATCHDOG(3)NAME
watchdog - watchdog Documentation
Python API library and shell utilities to monitor file system events.
DIRECTORY MONITORING MADE EASY WITH
· A cross-platform API.
· A shell tool to run commands in response to directory changes.
Get started quickly with a simple example in quickstart.
EASY INSTALLATION
You can use pip to install watchdog quickly and easily:
$ pip install watchdog
Need more help with installing? See installation.
INSTALLATION
watchdog requires Python 2.6 or above to work. If you are using a
Linux/FreeBSD/Mac OS X system, you already have Python installed. How‐
ever, you may wish to upgrade your system to Python 2.7 at least,
because this version comes with updates that can reduce compatibility
problems. See a list of Dependencies.
Installing from PyPI using pip
$ pip install watchdog
Installing from source tarballs
$ wget -c http://pypi.python.org/packages/source/w/watchdog/watchdog-0.8.3.tar.gz
$ tar zxvf watchdog-0.8.3.tar.gz
$ cd watchdog-0.8.3
$ python setup.py install
Installing from the code repository
$ git clone --recursive git://github.com/gorakhargosh/watchdog.git
$ cd watchdog
$ python setup.py install
Dependencies
watchdog depends on many libraries to do its job. The following is a
list of dependencies you need based on the operating system you are
using.
┌────────────────┬─────────┬───────────┬────────────────────┬─────┐
│Operating │ Windows │ Linux 2.6 │ │ BSD │
│system Depen‐ │ │ │ Mac OS │ │
│dency (row) │ │ │ X/ Dar‐ │ │
│ │ │ │ win │ │
├────────────────┼─────────┼───────────┼────────────────────┼─────┤
│XCode │ │ │ Yes │ │
├────────────────┼─────────┼───────────┼────────────────────┼─────┤
│PyYAML │ Yes │ Yes │ Yes │ Yes │
├────────────────┼─────────┼───────────┼────────────────────┼─────┤
│argh │ Yes │ Yes │ Yes │ Yes │
├────────────────┼─────────┼───────────┼────────────────────┼─────┤
│argparse │ Yes │ Yes │ Yes │ Yes │
└────────────────┴─────────┴───────────┴────────────────────┴─────┘
│select_backport │ │ │ Yes │ Yes │
│(Python 2.6) │ │ │ │ │
├────────────────┼─────────┼───────────┼────────────────────┼─────┤
│pathtools │ Yes │ Yes │ Yes │ Yes │
└────────────────┴─────────┴───────────┴────────────────────┴─────┘
Installing Dependencies
The watchmedo script depends on PyYAML which links with LibYAML. On
Mac OS X, you can use homebrew to install LibYAML:
brew install libyaml
On Linux, use your favorite package manager to install LibYAML. Here's
how you do it on Ubuntu:
sudo aptitude install libyaml-dev
On Windows, please install PyYAML using the binaries they provide.
Supported Platforms (and Caveats)
watchdog uses native APIs as much as possible falling back to polling
the disk periodically to compare directory snapshots only when it can‐
not use an API natively-provided by the underlying operating system.
The following operating systems are currently supported:
WARNING:
Differences between behaviors of these native API are noted below.
Linux 2.6+
Linux kernel version 2.6 and later come with an API called
inotify that programs can use to monitor file system events.
NOTE:
On most systems the maximum number of watches that can be
created per user is limited to 8192. watchdog needs one per
directory to monitor. To change this limit, edit
/etc/sysctl.conf and add:
fs.inotify.max_user_watches=16384
Mac OS X
The Darwin kernel/OS X API maintains two ways to monitor direc‐
tories for file system events:
· kqueue
· FSEvents
watchdog can use whichever one is available, preferring FSEvents
over kqueue(2). kqueue(2) uses open file descriptors for moni‐
toring and the current implementation uses Mac OS X File System
Monitoring Performance Guidelines to open these file descriptors
only to monitor events, thus allowing OS X to unmount volumes
that are being watched without locking them.
NOTE:
More information about how watchdog uses kqueue(2) is noted
in BSD Unix variants. Much of this information applies to Mac
OS X as well.
BSD variants come with kqueue which programs can use to monitor
changes to open file descriptors. Because of the way kqueue(2)
works, watchdog needs to open these files and directories in
read-only non-blocking mode and keep books about them.
watchdog will automatically open file descriptors for all new
files/directories created and close those for which are deleted.
NOTE:
The maximum number of open file descriptor per process limit
on your operating system can hinder watchdog's ability to
monitor files.
You should ensure this limit is set to at least 1024 (or a
value suitable to your usage). The following command appended
to your ~/.profile configuration file does this for you:
ulimit -n 1024
Windows Vista and later
The Windows API provides the ReadDirectoryChangesW. watchdog
currently contains implementation for a synchronous approach
requiring additional API functionality only available in Windows
Vista and later.
NOTE:
Since renaming is not the same operation as movement on Win‐
dows, watchdog tries hard to convert renames to movement
events. Also, because the ReadDirectoryChangesW API function
returns rename/movement events for directories even before
the underlying I/O is complete, watchdog may not be able to
completely scan the moved directory in order to successfully
queue movement events for files and directories within it.
NOTE:
Since the Windows API does not provide information about
whether an object is a file or a directory, delete events for
directories may be reported as a file deleted event.
OS Independent Polling
watchdog also includes a fallback-implementation that polls
watched directories for changes by periodically comparing snap‐
shots of the directory tree.
QUICKSTART
Below we present a simple example that monitors the current directory
recursively (which means, it will traverse any sub-directories) to
detect changes. Here is what we will do with the API:
1. Create an instance of the watchdog.observers.Observer thread class.
2. Implement a subclass of watchdog.events.FileSystemEventHandler (or
as in our case, we will use the built-in watchdog.events.Loggin‐
gEventHandler, which already does).
3. Schedule monitoring a few paths with the observer instance attaching
the event handler.
4. Start the observer thread and wait for it generate events without
blocking our main thread.
By default, an watchdog.observers.Observer instance will not monitor
sub-directories. By passing recursive=True in the call to watch‐
dog.observers.Observer.schedule() monitoring entire directory trees is
ensured.
A Simple Example
The following example program will monitor the current directory recur‐
sively for file system changes and simply log them to the console:
import sys
import time
import logging
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO,
format='%(asctime)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
path = sys.argv[1] if len(sys.argv) > 1 else '.'
event_handler = LoggingEventHandler()
observer = Observer()
observer.schedule(event_handler, path, recursive=True)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()observer.join()
To stop the program, press Control-C.
API REFERENCE
watchdog.events
module watchdog.events
synopsis
File system events and event handlers.
author yesudeep@google.com (Yesudeep Mangalapilly)
Event Classes
class watchdog.events.FileSystemEvent(src_path)
Bases: object
Immutable type that represents a file system event that is trig‐
gered when a change occurs on the monitored file system.
All FileSystemEvent objects are required to be immutable and
hence can be used as keys in dictionaries or be added to sets.
event_type = None
The type of the event as a string.
is_directory = False
True if event was emitted for a directory; False other‐
wise.
src_path
Source path of the file system object that triggered this
event.
class watchdog.events.FileSystemMovedEvent(src_path, dest_path)
Bases: watchdog.events.FileSystemEvent
File system event representing any kind of file system movement.
dest_path
The destination path of the move event.
class watchdog.events.FileMovedEvent(src_path, dest_path)
Bases: watchdog.events.FileSystemMovedEvent
File system event representing file movement on the file system.
class watchdog.events.DirMovedEvent(src_path, dest_path)
Bases: watchdog.events.FileSystemMovedEvent
File system event representing directory movement on the file
system.
class watchdog.events.FileModifiedEvent(src_path)
Bases: watchdog.events.FileSystemEvent
File system event representing file modification on the file
system.
class watchdog.events.DirModifiedEvent(src_path)
Bases: watchdog.events.FileSystemEvent
File system event representing directory modification on the
file system.
class watchdog.events.FileCreatedEvent(src_path)
Bases: watchdog.events.FileSystemEvent
File system event representing file creation on the file system.
class watchdog.events.DirCreatedEvent(src_path)
Bases: watchdog.events.FileSystemEvent
File system event representing directory creation on the file
system.
class watchdog.events.FileDeletedEvent(src_path)
Bases: watchdog.events.FileSystemEvent
File system event representing file deletion on the file system.
class watchdog.events.DirDeletedEvent(src_path)
Bases: watchdog.events.FileSystemEvent
File system event representing directory deletion on the file
system.
Event Handler Classes
class watchdog.events.FileSystemEventHandler
Bases: object
Base file system event handler that you can override methods
from.
dispatch(event)
Dispatches events to the appropriate methods.
Parameters
event (FileSystemEvent) -- The event object repre‐
senting the file system event.
on_any_event(event)
Catch-all event handler.
Parameters
event (FileSystemEvent) -- The event object repre‐
senting the file system event.
on_created(event)
Called when a file or directory is created.
Parameters
event (DirCreatedEvent or FileCreatedEvent) --
Event representing file/directory creation.
on_deleted(event)
Called when a file or directory is deleted.
Parameters
event (DirDeletedEvent or FileDeletedEvent) --
Event representing file/directory deletion.
on_modified(event)
Called when a file or directory is modified.
Parameters
event (DirModifiedEvent or FileModifiedEvent) --
Event representing file/directory modification.
on_moved(event)
Called when a file or a directory is moved or renamed.
Parameters
event (DirMovedEvent or FileMovedEvent) -- Event
representing file/directory movement.
class watchdog.events.PatternMatchingEventHandler(patterns=None,
ignore_patterns=None, ignore_directories=False, case_sensitive=False)
Bases: watchdog.events.FileSystemEventHandler
Matches given patterns with file paths associated with occurring
events.
case_sensitive
(Read-only) True if path names should be matched sensi‐
tive to case; False otherwise.
dispatch(event)
Dispatches events to the appropriate methods.
Parameters
event (FileSystemEvent) -- The event object repre‐
senting the file system event.
ignore_directories
(Read-only) True if directories should be ignored; False
otherwise.
ignore_patterns
(Read-only) Patterns to ignore matching event paths.
patterns
(Read-only) Patterns to allow matching event paths.
class watchdog.events.RegexMatchingEventHandler(regexes=['.*'],
ignore_regexes=[], ignore_directories=False, case_sensitive=False)
Bases: watchdog.events.FileSystemEventHandler
Matches given regexes with file paths associated with occurring
events.
case_sensitive
(Read-only) True if path names should be matched sensi‐
tive to case; False otherwise.
dispatch(event)
Dispatches events to the appropriate methods.
Parameters
event (FileSystemEvent) -- The event object repre‐
senting the file system event.
ignore_directories
(Read-only) True if directories should be ignored; False
otherwise.
ignore_regexes
(Read-only) Regexes to ignore matching event paths.
regexes
(Read-only) Regexes to allow matching event paths.
class watchdog.events.LoggingEventHandler
Bases: watchdog.events.FileSystemEventHandler
Logs all the events captured.
watchdog.observers.api
Immutables
class watchdog.observers.api.ObservedWatch(path, recursive)
Bases: object
An scheduled watch.
Parameters
· path -- Path string.
· recursive -- True if watch is recursive; False other‐
wise.
is_recursive
Determines whether subdirectories are watched for the
path.
path The path that this watch monitors.
Collections
class watchdog.observers.api.EventQueue(maxsize=0)
Bases: watchdog.utils.bricks.SkipRepeatsQueue
Thread-safe event queue based on a special queue that skips
adding the same event (FileSystemEvent) multiple times consecu‐
tively. Thus avoiding dispatching multiple event handling calls
when multiple identical events are produced quicker than an
observer can consume them.
Classes
class watchdog.observers.api.EventEmitter(event_queue, watch, time‐
out=1)
Bases: watchdog.utils.BaseThread
Producer thread base class subclassed by event emitters that
generate events and populate a queue with them.
Parameters
· event_queue (watchdog.events.EventQueue) -- The event
queue to populate with generated events.
· watch (ObservedWatch) -- The watch to observe and pro‐
duce events for.
· timeout (float) -- Timeout (in seconds) between succes‐
sive attempts at reading events.
queue_event(event)
Queues a single event.
Parameters
event (An instance of
watchdog.events.FileSystemEvent or a subclass.) --
Event to be queued.
queue_events(timeout)
Override this method to populate the event queue with
events per interval period.
Parameters
timeout (float) -- Timeout (in seconds) between
successive attempts at reading events.
timeout
Blocking timeout for reading events.
watch The watch associated with this emitter.
class watchdog.observers.api.EventDispatcher(timeout=1)
Bases: watchdog.utils.BaseThread
Consumer thread base class subclassed by event observer threads
that dispatch events from an event queue to appropriate event
handlers.
Parameters
timeout (float) -- Event queue blocking timeout (in sec‐
onds).
dispatch_events(event_queue, timeout)
Override this method to consume events from an event
queue, blocking on the queue for the specified timeout
before raising queue.Empty.
Parameters
· event_queue (EventQueue) -- Event queue to popu‐
late with one set of events.
· timeout (float) -- Interval period (in seconds)
to wait before timing out on the event queue.
Raises queue.Empty
event_queue
The event queue which is populated with file system
events by emitters and from which events are dispatched
by a dispatcher thread.
timeout
Event queue block timeout.
class watchdog.observers.api.BaseObserver(emitter_class, timeout=1)
Bases: watchdog.observers.api.EventDispatcher
Base observer.
add_handler_for_watch(event_handler, watch)
Adds a handler for the given watch.
Parameters
· event_handler (‐
watchdog.events.FileSystemEventHandler or a sub‐
class) -- An event handler instance that has
appropriate event handling methods which will be
called by the observer in response to file sys‐
tem events.
· watch (An instance of ObservedWatch or a sub‐
class of ObservedWatch) -- The watch to add a
handler for.
emitters
Returns event emitter created by this observer.
remove_handler_for_watch(event_handler, watch)
Removes a handler for the given watch.
Parameters
· event_handler (‐
watchdog.events.FileSystemEventHandler or a sub‐
class) -- An event handler instance that has
appropriate event handling methods which will be
called by the observer in response to file sys‐
tem events.
· watch (An instance of ObservedWatch or a sub‐
class of ObservedWatch) -- The watch to remove a
handler for.
schedule(event_handler, path, recursive=False)
Schedules watching a path and calls appropriate methods
specified in the given event handler in response to file
system events.
Parameters
· event_handler (‐
watchdog.events.FileSystemEventHandler or a sub‐
class) -- An event handler instance that has
appropriate event handling methods which will be
called by the observer in response to file sys‐
tem events.
· path (str) -- Directory path that will be moni‐
tored.
· recursive (bool) -- True if events will be emit‐
ted for sub-directories traversed recursively;
False otherwise.
Returns
An ObservedWatch object instance representing a
watch.
unschedule(watch)
Unschedules a watch.
Parameters
watch (An instance of ObservedWatch or a subclass
of ObservedWatch) -- The watch to unschedule.
unschedule_all()
Unschedules all watches and detaches all associated event
handlers.
watchdog.observers
module watchdog.observers
synopsis
Observer that picks a native implementation if available.
author yesudeep@google.com (Yesudeep Mangalapilly)
Classes
watchdog.observers.Observer
alias of InotifyObserver
Observer thread that schedules watching directories and dispatches
calls to event handlers.
You can also import platform specific classes directly and use it
instead of Observer. Here is a list of implemented observer classes.:
┌────────────────────────┬─────────────────────┬─────────────────────┐
│Class │ Platforms │ Note │
├────────────────────────┼─────────────────────┼─────────────────────┤
│inotify.InotifyOb‐ │ Linux 2.6.13+ │ inotify(7) based │
│server │ │ observer │
├────────────────────────┼─────────────────────┼─────────────────────┤
│fsevents.FSEv‐ │ Mac OS X │ FSEvents based │
│entsObserver │ │ observer │
├────────────────────────┼─────────────────────┼─────────────────────┤
│kqueue.KqueueOb‐ │ Mac OS X and BSD │ kqueue(2) based │
│server │ with kqueue(2) │ observer │
├────────────────────────┼─────────────────────┼─────────────────────┤
│read_direc‐ │ MS Windows │ Windows API-based │
│tory_changes.Win‐ │ │ observer │
│dowsApiObserver │ │ │
├────────────────────────┼─────────────────────┼─────────────────────┤
│polling.PollingObserver │ Any │ fallback implemen‐ │
│ │ │ tation │
└────────────────────────┴─────────────────────┴─────────────────────┘
watchdog.observers.polling
module watchdog.observers.polling
synopsis
Polling emitter implementation.
author yesudeep@google.com (Yesudeep Mangalapilly)
Classes
class watchdog.observers.polling.PollingObserver(timeout=1)
Bases: watchdog.observers.api.BaseObserver
Platform-independent observer that polls a directory to detect
file system changes.
class watchdog.observers.polling.PollingObserverVFS(stat, listdir,
polling_interval=1)
Bases: watchdog.observers.api.BaseObserver
File system independent observer that polls a directory to
detect changes.
__init__(stat, listdir, polling_interval=1)
Parameters
· stat -- stat function. See os.stat for details.
· listdir -- listdir function. See os.listdir for
details.
· polling_interval (float) -- interval in seconds
between polling the file system.
watchdog.utils
module watchdog.utils
synopsis
Utility classes and functions.
author yesudeep@google.com (Yesudeep Mangalapilly)
Classes
class watchdog.utils.BaseThread
Bases: threading.Thread
Convenience class for creating stoppable threads.
daemon A boolean value indicating whether this thread is a dae‐
mon thread (True) or not (False).
This must be set before start() is called, otherwise Run‐
timeError is raised. Its initial value is inherited from
the creating thread; the main thread is not a daemon
thread and therefore all threads created in the main
thread default to daemon = False.
The entire Python program exits when no alive non-daemon
threads are left.
ident Thread identifier of this thread or None if it has not
been started.
This is a nonzero integer. See the thread.get_ident()
function. Thread identifiers may be recycled when a
thread exits and another thread is created. The identi‐
fier is available even after the thread has exited.
isAlive()
Return whether the thread is alive.
This method returns True just before the run() method
starts until just after the run() method terminates. The
module function enumerate() returns a list of all alive
threads.
is_alive()
Return whether the thread is alive.
This method returns True just before the run() method
starts until just after the run() method terminates. The
module function enumerate() returns a list of all alive
threads.
join(timeout=None)
Wait until the thread terminates.
This blocks the calling thread until the thread whose
join() method is called terminates -- either normally or
through an unhandled exception or until the optional
timeout occurs.
When the timeout argument is present and not None, it
should be a floating point number specifying a timeout
for the operation in seconds (or fractions thereof). As
join() always returns None, you must call isAlive() after
join() to decide whether a timeout happened -- if the
thread is still alive, the join() call timed out.
When the timeout argument is not present or None, the
operation will block until the thread terminates.
A thread can be join()ed many times.
join() raises a RuntimeError if an attempt is made to
join the current thread as that would cause a deadlock.
It is also an error to join() a thread before it has been
started and attempts to do so raises the same exception.
name A string used for identification purposes only.
It has no semantics. Multiple threads may be given the
same name. The initial name is set by the constructor.
on_thread_start()
Override this method instead of start(). start() calls
this method.
This method is called right before this thread is started
and this object’s run() method is invoked.
on_thread_stop()
Override this method instead of stop(). stop() calls
this method.
This method is called immediately after the thread is
signaled to stop.
run() Method representing the thread's activity.
You may override this method in a subclass. The standard
run() method invokes the callable object passed to the
object's constructor as the target argument, if any, with
sequential and keyword arguments taken from the args and
kwargs arguments, respectively.
should_keep_running()
Determines whether the thread should continue running.
stop() Signals the thread to stop.
watchdog.utils.dirsnapshot
module watchdog.utils.dirsnapshot
synopsis
Directory snapshots and comparison.
author yesudeep@google.com (Yesudeep Mangalapilly)
Where are the moved events? They
This implementation does not take partition boundaries into
consideration. It will only work when the directory tree is
entirely on the same file system. More specifically, any part
of the code that depends on inode numbers can break if parti‐
tion boundaries are crossed. In these cases, the snapshot
diff will represent file/directory movement as created and
deleted events.
Classes
class watchdog.utils.dirsnapshot.DirectorySnapshot(path, recur‐
sive=True, walker_callback=<function <lambda>>, stat=<built-in function
stat>, listdir=<built-in function listdir>)
Bases: object
A snapshot of stat information of files in a directory.
Parameters
· path (str) -- The directory path for which a snapshot
should be taken.
· recursive (bool) -- True if the entire directory tree
should be included in the snapshot; False otherwise.
· walker_callback --
Deprecated since version 0.7.2.
· stat --
Use custom stat function that returns a stat structure
for path. Currently only st_dev, st_ino, st_mode and
st_mtime are needed.
A function with the signature walker_callback(path,
stat_info) which will be called for every entry in the
directory tree.
· listdir -- Use custom listdir function. See os.listdir
for details.
inode(path)
Returns an id for path.
path(id)
Returns path for id. None if id is unknown to this snap‐
shot.
paths Set of file/directory paths in the snapshot.
stat_info(path)
Returns a stat information object for the specified path
from the snapshot.
Attached information is subject to change. Do not use
unless you specify stat in constructor. Use inode(),
mtime(), isdir() instead.
Parameters
path -- The path for which stat information should
be obtained from a snapshot.
class watchdog.utils.dirsnapshot.DirectorySnapshotDiff(ref, snapshot)
Bases: object
Compares two directory snapshots and creates an object that rep‐
resents the difference between the two snapshots.
Parameters
· ref (DirectorySnapshot) -- The reference directory
snapshot.
· snapshot (DirectorySnapshot) -- The directory snapshot
which will be compared with the reference snapshot.
dirs_created
List of directories that were created.
dirs_deleted
List of directories that were deleted.
dirs_modified
List of directories that were modified.
dirs_moved
List of directories that were moved.
Each event is a two-tuple the first item of which is the
path that has been renamed to the second item in the
tuple.
files_created
List of files that were created.
files_deleted
List of files that were deleted.
files_modified
List of files that were modified.
files_moved
List of files that were moved.
Each event is a two-tuple the first item of which is the
path that has been renamed to the second item in the
tuple.
CONTRIBUTING
Welcome hacker! So you have got something you would like to see in
watchdog? Whee. This document will help you get started.
Important URLs
watchdog uses git to track code history and hosts its code repository
at github. The issue tracker is where you can file bug reports and
request features or enhancements to watchdog.
Before you start
Ensure your system has the following programs and libraries installed
before beginning to hack:
1. Python
2. git
3. ssh
4. XCode (on Mac OS X)
5. select_backport (on BSD/Mac OS X if you're using Python 2.6)
Setting up the Work Environment
watchdog makes extensive use of zc.buildout to set up its work environ‐
ment. You should get familiar with it.
Steps to setting up a clean environment:
1. Fork the code repository into your github account. Let us call you
hackeratti for the sake of this example. Replace hackeratti with
your own username below.
2. Clone your fork and setup your environment:
$ git clone --recursive git@github.com:hackeratti/watchdog.git
$ cd watchdog
$ python tools/bootstrap.py --distribute
$ bin/buildout
IMPORTANT:
Re-run bin/buildout every time you make a change to the buildout.cfg
file.
That's it with the setup. Now you're ready to hack on watchdog.
Enabling Continuous Integration
The repository checkout contains a script called autobuild.sh which you
must run prior to making changes. It will detect changes to Python
source code or restructuredText documentation files anywhere in the
directory tree and rebuild sphinx documentation, run all tests using
nose, and generate coverage reports.
Start it by issuing this command in the watchdog directory checked out
earlier:
$ tools/autobuild.sh
...
Happy hacking!
Found a bug in or want a feature added to watchdog? You can fork the
official code repository or file an issue ticket at the issue tracker.
You can also ask questions at the official mailing list. You may also
want to refer to hacking for information about contributing code or
documentation to watchdog.
· genindex
· modindex
· search
AUTHOR
Yesudeep Mangalapilly
COPYRIGHT
2010, Yesudeep Mangalapilly
0.8.3 October 20, 2015 WATCHDOG(3)
