Repository: kivy/kivy

Issue:
Clock.schedule_interval more limited than expected
Hello,

Many thanks to the kivy team for building kivy!

I tried to use Clock.schedule_interval to set a timer to record some data in my Kivy application.
Found some limits, I ask users here:
https://groups.google.com/forum/#!topic/kivy-users/3Iubhg6nhvo

Seems that whatever we set, the smallest dt is around 0.01283s
I could not understand where from this comes, tried to change the maxfps without any success.

Requesting for a period of 0.1s brings (on my PC) an average dt of 0.1027s.
That's not very precise.

Maybe, should write something in the documentation about the limits.

For people running into same problem, can use Timer object from the standard lib, ex:
http://stackoverflow.com/questions/12435211/python-threading-timer-repeat-function-every-n-seconds



Discussion:
I guess it should be documented that kivy's clock always returns time not as the actual time but the frame time which tends to be a function of the frame rate.

Will be perfect to see it in the documentation.
I used it as it is, thinking that a 100Hz timer is pretty easy to achieve, today, for a computer. But actually, I got only ~80Hz, running into this limitation.

By rolling my own kivy clock, using event timeouts, I am able to get 1kHz. But for games it's important to get a predicable clock hence the default kivy behavior.

Do you have a sample program that demonstrates the issue:

I can get a kivent game with framerate unlocked to run at 3000+ fps, clock is definitely capable of running at an even smaller dt


Resolving patch:
diff --git a/kivy/clock.py b/kivy/clock.py
--- a/kivy/clock.py
+++ b/kivy/clock.py
@@ -217,9 +217,139 @@ def my_callback(dt):
 callback is now significantly slower and highly discouraged. Instead, the
 returned events should be used to cancel. As a tradeoff, all the other methods
 are now significantly faster than before.
+
+Advanced Clock Details
+-----------------------
+
+The following section goes into the internal kivy clock details as well
+as the various clock options. It is meant only for advanced users.
+
+Fundamentally, the Kivy clock attempts to execute any scheduled callback
+rhythmically as determined by the specified fps (frame per second, see
+``maxfps`` in :mod:`~kivy.config`). That is, ideally, given e.g. a desired fps
+of 30, the clock will execute the callbacks at intervals of 1 / 30 seconds, or
+every 33.33 ms. All the callbacks in a frame are given the same timestamp,
+i.e. the ``dt`` passed to the callback are all the same and it's the difference
+in time between the start of this and the previous frame.
+
+Because of inherent indeterminism, the frames do not actually occur exactly
+at intervals of the fps and ``dt`` may be under or over the desired fps.
+Also, once the timeout is "close enough" to the desired timeout, as determined
+internally, Kivy will execute the callback in the current frame even when the
+"actual time" has not elapsed the ``timeout`` amount.
+
+Kivy offers now, since ``1.9.2``, multiple clocks with different behaviors.
+
+Default Clock
+^^^^^^^^^^^^^^
+
+The default clock (``default``) behaves as described above. When a callback
+with a timeout of zero or non-zero is scheduled, they are executed at the frame
+that is near the timeout, which is a function of the fps. So a timeout of zero
+would still result in a delay of one frame or about 1 / fps, typically a bit
+less but sometimes more depending on the CPU usage of the other events
+scheduled for that frame.
+
+In a test using a fps of 30, a callback with a timeout of 0, 0.001, and 0.05,
+resulted in a mean callback delay of 0.02487, 0.02488, and 0.05011 seconds,
+respectively. When tested with a fps of 600 the delay for 0.05 was similar,
+except the standard deviation was reduced resulting in overall better accuracy.
+
+Interruptible Clock
+^^^^^^^^^^^^^^^^^^^^
+
+The default clock suffers from the quantization problem, as frames occur only
+on intervals and any scheduled timeouts will not be able to occur during an
+interval. For example, with the timeout of 0.05, while the mean was 0.05011,
+its values ranged between 0.02548 - 0.07348 and a standard deviation of 0.002.
+Also, there's the minimum timeout of about 0.02487.
+
+The interruptible clock (``interrupt``) will execute timeouts even during a
+frame. So a timeout of zero will execute as quickly as possible and similarly
+a non-zero timeout will be executed even during the interval.
+
+This clock, and all the clocks described after this have an option,
+:attr:`ClockBaseInterruptBehavior.interupt_next_only`. When True, any of the
+behavior new behavior will only apply to the callbacks with a timeout of
+zero. Non-zero timeouts will behave like in the default clock. E.g. for this
+clock when True, only zero timeouts will execute during the the interval.
+
+In a test using a fps of 30, a callback with a timeout of 0, 0.001, and 0.05,
+resulted in a mean callback delay of 0.00013, 0.00013, and 0.04120 seconds,
+respectively when :attr:`ClockBaseInterruptBehavior.interupt_next_only` was
+False. Also, compared to the default clock the standard deviation was reduced.
+When :attr:`ClockBaseInterruptBehavior.interupt_next_only` was True, the values
+were 0.00010, 0.02414, and 0.05034, respectively.
+
+Free Clock
+^^^^^^^^^^^
+
+The interruptible clock may not be ideal for all cases because all the events
+are executed during the intervals and events are not executed anymore
+rhythmically as multiples of the fps. For example, there may not be any benefit
+for the graphics to update in a sub-interval, so the additional accuracy
+wastes CPU.
+
+The Free clock (``free_all``) solves this by having ``Clock.xxx_free`` versions
+of all the Clock scheduling methods. By free, we mean the event is free from
+the fps because it's not fps limited. E.g.
+:meth:`CyClockBaseFree.create_trigger_free` corresponds to
+:meth:`CyClockBase.create_trigger`. Only when an event scheduled using the
+``Clock.xxx_free`` methods is present will the clock interrupt and execute
+the events during the interval. So, if no ``free`` event is present the clock
+behaves like the ``default`` clock, otherwise it behaves like the ``interrupt``
+clock.
+
+In a test using a fps of 30, a callback with a timeout of 0s, 0.001s, and
+0.05s, resulted in a mean callback delay of 0.00012s, 0.00017s, and 0.04121s
+seconds, respectively when it was a free event and 0.02403s, 0.02405s, and
+0.04829s, respectively when it wasn't.
+
+Free Only Clock
+^^^^^^^^^^^^^^^^^
+
+The Free clock executes all events when a free event was scheduled. This
+results in normal events also being execute in the middle of the interval
+when a free event is scheduled. For example, above, when a free event was
+absent, a normal event with a 0.001s timeout was delayed for 0.02405s. However,
+if a free event happened to be also scheduled, the normal event was only
+delayed 0.00014s, which may be undesirable.
+
+The Free only clock (``free_only``) solves it by only executing free events
+during the interval and normal events are always executed like with the
+default clock. For example, in the presence of a free event, a normal event
+with a timeout of 0.001s still had a delay of 0.02406. So this clock,
+treats free and normal events independently, with normal events always being
+fps limited, but never the free events.
+
+Summary
+^^^^^^^^
+
+The kivy clock type to use can be set with the ``kivy_clock`` option the
+:mod:`~kivy.config`. If ``KIVY_CLOCK`` is present in the environment it
+overwrites the config selection. Its possible values are as follows:
+
+* When ``kivy_clock`` is ``default``, the normal clock, :class:`ClockBase`,
+  which limits callbacks to the maxfps quantization - is used.
+* When ``kivy_clock`` is ``interrupt``, a interruptible clock,
+  :class:`ClockBaseInterrupt`, which doesn't limit any callbacks to the
+  maxfps - is used. Callbacks will be executed at any time.
+* When ``kivy_clock`` is ``free_all``, a interruptible clock,
+  :class:`ClockBaseFreeInterruptAll`, which doesn't limit any callbacks to the
+  maxfps in the presence of free events, but in their absence it limits events
+  to the fps quantization interval - is used.
+* When ``kivy_clock`` is ``free_only``, a interruptible clock,
+  :class:`ClockBaseFreeInterruptAll`, which treats free and normal events
+  independently; normal events are fps limited while free events are not - is
+  used.
+
 '''
 
-__all__ = ('Clock', 'CyClockBase', 'ClockBase', 'ClockEvent', 'mainthread')
+__all__ = (
+    'Clock', 'ClockEvent', 'FreeClockEvent', 'CyClockBase', 'CyClockBaseFree',
+    'ClockBaseBehavior', 'ClockBaseInterruptBehavior',
+    'ClockBaseInterruptFreeBehavior', 'ClockBase', 'ClockBaseInterrupt',
+    'ClockBaseFreeInterruptAll', 'ClockBaseFreeInterruptOnly', 'mainthread')
 
 from sys import platform
 from os import environ
@@ -228,10 +358,18 @@ def my_callback(dt):
 from kivy.weakmethod import WeakMethod
 from kivy.config import Config
 from kivy.logger import Logger
-from kivy.compat import clock as _default_time
+from kivy.compat import clock as _default_time, PY2
 import time
 from threading import Lock
-from kivy._clock import CyClockBase, ClockEvent
+from kivy._clock import CyClockBase, ClockEvent, FreeClockEvent, \
+    CyClockBaseFree
+try:
+    from multiprocessing import Event as MultiprocessingEvent
+except ImportError:  # https://bugs.python.org/issue3770
+    from threading import Event as MultiprocessingEvent
+from threading import Event as ThreadingEvent
+
+# some reading: http://gameprogrammingpatterns.com/game-loop.html
 
 
 def _get_sleep_obj():
@@ -317,8 +455,8 @@ def _usleep(microseconds, obj=None):
         time.sleep(microseconds / 1000000.)
 
 
-class ClockBase(CyClockBase):
-    '''A clock object with event support.
+class ClockBaseBehavior(object):
+    '''The base of the kivy clock.
     '''
 
     _dt = 0.0001
@@ -330,16 +468,25 @@ class ClockBase(CyClockBase):
     _rfps_counter = 0
     _frames = 0
     _frames_displayed = 0
-    _max_fps = 30.
-    _sleep_obj = None
+    _events_duration = 0
+    '''The measured time that it takes to process all the events etc, excepting
+    any sleep or waiting time. It is the average and is updated every 5
+    seconds.
+    '''
+
+    _duration_count = 0
+    _sleep_time = 0
+    _duration_ts0 = 0
 
     MIN_SLEEP = 0.005
+    '''The minimum time to sleep. If the remaining time is less than this,
+    the event loop will continuo
+    '''
     SLEEP_UNDERSHOOT = MIN_SLEEP - 0.001
 
-    def __init__(self):
-        super(ClockBase, self).__init__()
-        self._sleep_obj = _get_sleep_obj()
-        self._start_tick = self._last_tick = self.time()
+    def __init__(self, **kwargs):
+        super(ClockBaseBehavior, self).__init__(**kwargs)
+        self._duration_ts0 = self._start_tick = self._last_tick = self.time()
         self._max_fps = float(Config.getint('graphics', 'maxfps'))
 
     @property
@@ -369,7 +516,31 @@ def frames_displayed(self):
     def usleep(self, microseconds):
         '''Sleeps for the number of microseconds.
         '''
-        _usleep(microseconds, self._sleep_obj)
+        pass
+
+    def idle(self):
+        '''(internal) waits here until the next frame.
+        '''
+        fps = self._max_fps
+        if fps > 0:
+            min_sleep = self.get_resolution()
+            undershoot = 4 / 5. * min_sleep
+            usleep = self.usleep
+            ready = self._check_ready
+
+            done, sleeptime = ready(fps, min_sleep, undershoot)
+            while not done:
+                usleep(1000000 * sleeptime)
+                done, sleeptime = ready(fps, min_sleep, undershoot)
+
+        current = self.time()
+        self._dt = current - self._last_tick
+        self._last_tick = current
+        return current
+
+    def _check_ready(self, fps, min_sleep, undershoot):
+        sleeptime = 1 / fps - (self.time() - self._last_tick)
+        return sleeptime - undershoot <= min_sleep, sleeptime - undershoot
 
     def tick(self):
         '''Advance the clock to the next step. Must be called every frame.
@@ -378,24 +549,22 @@ def tick(self):
 
         self._release_references()
 
-        # do we need to sleep ?
-        if self._max_fps > 0:
-            min_sleep = self.MIN_SLEEP
-            sleep_undershoot = self.SLEEP_UNDERSHOOT
-            fps = self._max_fps
-            usleep = self.usleep
-
-            sleeptime = 1 / fps - (self.time() - self._last_tick)
-            while sleeptime - sleep_undershoot > min_sleep:
-                usleep(1000000 * (sleeptime - sleep_undershoot))
-                sleeptime = 1 / fps - (self.time() - self._last_tick)
+        ts = self.time()
+        current = self.idle()
 
         # tick the current time
-        current = self.time()
-        self._dt = current - self._last_tick
         self._frames += 1
         self._fps_counter += 1
-        self._last_tick = current
+
+        # compute how long the event processing takes
+        self._duration_count += 1
+        self._sleep_time += current - ts
+        t_tot = current - self._duration_ts0
+        if t_tot >= 1.:
+            self._events_duration = \
+                (t_tot - self._sleep_time) / float(self._duration_count)
+            self._duration_ts0 = current
+            self._sleep_time = self._duration_count = 0
 
         # calculate fps things
         if self._last_fps_tick is None:
@@ -444,7 +613,165 @@ def get_boottime(self):
 
     time = staticmethod(partial(_default_time))
 
-ClockBase.time.__doc__ = '''Proxy method for :func:`~kivy.compat.clock`. '''
+ClockBaseBehavior.time.__doc__ = '''Proxy method for :func:`~kivy.compat.clock`. '''
+
+
+class ClockBaseInterruptBehavior(ClockBaseBehavior):
+    '''A kivy clock which can be interrupted during a frame to execute events.
+    '''
+
+    interupt_next_only = False
+    _event = None
+    _get_min_timeout_func = None
+
+    def __init__(self, interupt_next_only=False, **kwargs):
+        super(ClockBaseInterruptBehavior, self).__init__(**kwargs)
+        self._event = MultiprocessingEvent() if PY2 else ThreadingEvent()
+        self.interupt_next_only = interupt_next_only
+        self._get_min_timeout_func = self.get_min_timeout
+
+    def usleep(self, microseconds):
+        self._event.clear()
+        self._event.wait(microseconds / 1000000.)
+
+    def on_schedule(self, event):
+        fps = self._max_fps
+        if not fps:
+            return
+
+        if not event.timeout or (
+                not self.interupt_next_only and event.timeout
+                <= 1 / fps  # remaining time
+                - (self.time() - self._last_tick)  # elapsed time
+                + 4 / 5. * self.get_resolution()):  # resolution fudge factor
+            self._event.set()
+
+    def idle(self):
+        fps = self._max_fps
+        event = self._event
+        resolution = self.get_resolution()
+        if fps > 0:
+            done, sleeptime = self._check_ready(
+                fps, resolution, 4 / 5. * resolution)
+            if not done:
+                event.wait(sleeptime)
+
+        current = self.time()
+        self._dt = current - self._last_tick
+        self._last_tick = current
+        event.clear()
+        # anything scheduled from now on, if scheduled for the upcoming frame
+        # will cause a timeout of the event on the next idle due to on_schedule
+        # `self._last_tick = current` must happen before clear, otherwise the
+        # on_schedule computation is wrong when exec between the clear and
+        # the `self._last_tick = current` bytecode.
+        return current
+
+    def _check_ready(self, fps, min_sleep, undershoot):
+        if self._event.is_set():
+            return True, 0
+
+        t = self._get_min_timeout_func()
+        if not t:
+            return True, 0
+
+        if not self.interupt_next_only:
+            curr_t = self.time()
+            sleeptime = min(1 / fps - (curr_t - self._last_tick), t - curr_t)
+        else:
+            sleeptime = 1 / fps - (self.time() - self._last_tick)
+        return sleeptime - undershoot <= min_sleep, sleeptime - undershoot
+
+
+class ClockBaseInterruptFreeBehavior(ClockBaseInterruptBehavior):
+    '''A base class for the clock that interrupts the sleep interval for
+    free events.
+    '''
+
+    def __init__(self, **kwargs):
+        super(ClockBaseInterruptFreeBehavior, self).__init__(**kwargs)
+        self._get_min_timeout_func = self.get_min_free_timeout
+
+    def on_schedule(self, event):
+        if not event.free:  # only wake up for free events
+            return
+        # free events should use real time not frame time
+        event._last_dt = self.time()
+        return super(ClockBaseInterruptFreeBehavior,
+                     self).on_schedule(event)
+
+
+class ClockBase(ClockBaseBehavior, CyClockBase):
+    '''The ``default`` kivy clock. See module for details.
+    '''
+
+    _sleep_obj = None
+
+    def __init__(self, **kwargs):
+        super(ClockBase, self).__init__(**kwargs)
+        self._sleep_obj = _get_sleep_obj()
+
+    def usleep(self, microseconds):
+        _usleep(microseconds, self._sleep_obj)
+
+
+class ClockBaseInterrupt(ClockBaseInterruptBehavior, CyClockBase):
+    '''The ``interrupt`` kivy clock. See module for details.
+    '''
+
+    pass
+
+
+class ClockBaseFreeInterruptAll(
+        ClockBaseInterruptFreeBehavior, CyClockBaseFree):
+    '''The ``free_all`` kivy clock. See module for details.
+    '''
+
+    pass
+
+
+class ClockBaseFreeInterruptOnly(
+        ClockBaseInterruptFreeBehavior, CyClockBaseFree):
+    '''The ``free_only`` kivy clock. See module for details.
+    '''
+
+    def idle(self):
+        fps = self._max_fps
+        if fps > 0:
+            event = self._event
+            min_sleep = self.get_resolution()
+            usleep = self.usleep
+            undershoot = 4 / 5. * min_sleep
+            min_t = self.get_min_free_timeout
+            interupt_next_only = self.interupt_next_only
+
+            current = self.time()
+            sleeptime = 1 / fps - (current - self._last_tick)
+            while sleeptime - undershoot > min_sleep:
+                if event.is_set():
+                    do_free = True
+                else:
+                    t = min_t()
+                    if not t:
+                        do_free = True
+                    elif interupt_next_only:
+                        do_free = False
+                    else:
+                        sleeptime = min(sleeptime, t - current)
+                        do_free = sleeptime - undershoot <= min_sleep
+
+                if do_free:
+                    event.clear()
+                    self._process_free_events(current)
+                else:
+                    event.wait(sleeptime - undershoot)
+                current = self.time()
+                sleeptime = 1 / fps - (current - self._last_tick)
+
+        self._dt = current - self._last_tick
+        self._last_tick = current
+        event.clear()  # this needs to stay after _last_tick
+        return current
 
 
 def mainthread(func):
@@ -461,6 +788,7 @@ def callback(self, *args):
             print('The request succedded!',
                   'This callback is called in the main thread.')
 
+
         self.req = UrlRequest(url='http://...', on_success=callback)
 
     .. versionadded:: 1.8.0
@@ -473,7 +801,18 @@ def callback_func(dt):
     return delayed_func
 
 if 'KIVY_DOC_INCLUDE' in environ:
-    #: Instance of :class:`ClockBase`.
+    #: Instance of :class:`ClockBaseBehavior`.
     Clock = None
 else:
-    Clock = register_context('Clock', ClockBase)
+    _classes = {'default': ClockBase, 'interrupt': ClockBaseInterrupt,
+                'free_all': ClockBaseFreeInterruptAll,
+                'free_only': ClockBaseFreeInterruptOnly}
+    _clk = environ.get('KIVY_CLOCK', Config.get('kivy', 'kivy_clock'))
+    if _clk not in _classes:
+        raise Exception(
+            '{} is not a valid kivy clock. Valid clocks are {}'.format(
+                _clk, sorted(_classes.keys())))
+
+    Clock = register_context('Clock', _classes[_clk])
+    '''The kivy Clock instance. See module documentation for details.
+    '''
diff --git a/kivy/config.py b/kivy/config.py
--- a/kivy/config.py
+++ b/kivy/config.py
@@ -190,6 +190,8 @@
         :class:`~kivy.uix.behaviors.buttonbehavior.ButtonBehavior` to
         make sure they display their current visual state for the given
         time.
+    `kivy_clock`: one of `default`, `interrupt`, `free_all`, `free_only`
+        The clock type to use with kivy. See :mod:`kivy.clock`.
 
 :input:
 
@@ -263,6 +265,7 @@
 
 .. versionchanged:: 1.9.2
     `min_state_time` has been added to the `graphics` section.
+    `kivy_clock` has been added to the kivy section
 
 .. versionchanged:: 1.9.0
     `borderless` and `window_state` have been added to the graphics section.
@@ -309,7 +312,7 @@
 _is_rpi = exists('/opt/vc/include/bcm_host.h')
 
 # Version number of current configuration format
-KIVY_CONFIG_VERSION = 15
+KIVY_CONFIG_VERSION = 16
 
 Config = None
 '''The default Kivy configuration object. This is a :class:`ConfigParser`
@@ -799,6 +802,9 @@ def name(self, value):
         elif version == 14:
             Config.setdefault('graphics', 'min_state_time', '.035')
 
+        elif version == 15:
+            Config.setdefault('kivy', 'kivy_clock', 'default')
+
         # elif version == 1:
         #    # add here the command for upgrading from configuration 0 to 1
 


Test patch: