On 12-10-08 9:31 AM, Gabès Jean wrote:
>
> In shinken/objects/schedulingitem.py:
>
> >                  else:
> > -                    self.next_chk = int(now + time_add)
> > +                    self.next_chk = int(self.next_chk + time_add)
> > +                # If check interval is a multiple of 60 (seconds), the check must be scheduler
> > +                # between 0 et 49 absolute seconds.
> > +                # Else re-schedule if the check is scheduled between 50 et 60 inclusively in absolute seconds.
> > +                # We assume here that checks do not take more than 10 seconds to execute.
> But here the problem is not really teh scheduling, that is always 
> '58s' if I understand it well, but the data we give to graphite as the 
> time get. If we look back at the start of this ticket, the graphite 
> problem was that the scheduling was drifting, so we did have in the 
> behavior : "next check = last planned check + 5min + latency" (because 
> we used the check time arrival, and not the planned time). Now we are 
> using the last time, without looking at the real time where we 
> received the check (so we reduce the latency from the scheduling). Now 
> : is graphite able to manage this? Graphite problem was a constant 
> drifting, but now the value is nearly constant, and only one check 
> here or here can have 1~2s more (the latency is not possible to 
> compute by definition), so it should be ok for it to manage. Maybe the 
> 49 thing is just not need for graphite graphs to be ok. Can we test 
> with and without the 49 part to see if there are holes in the graphs? Jean
> … <#>
> On Mon, Oct 8, 2012 at 2:50 PM, xkilian <notifications@github.com> 
> wrote: In shinken/objects/schedulingitem.py: > else: > - self.next_chk 
> = int(now + time_add) > + self.next_chk = int(self.next_chk + 
> time_add) > + # If check interval is a multiple of 60 (seconds), the 
> check must be scheduler > + # between 0 et 49 absolute seconds. > + # 
> Else re-schedule if the check is scheduled between 50 et 60 
> inclusively in absolute seconds. > + # We assume here that checks do 
> not take more than 10 seconds to execute. It is the check latency… 
> Let's take my example series: here is the planned time and check 
> execution latency: 10 h 0 m 58s, latency 1s 10 h 1 m 58s, latency 3s 
> 10 h 2 m 58s, latency 1s 10 h 3 m 58s, latency 1s 10 h 4 m 58s, 
> latency 1s Graphite time-stamped data 10 h 0 m 59s 10 h 2 m 01s 10 h 2 
> m 59s 10 h 3 m 59s 10 h 4 m 59s That is why having the check scheduled 
> in the last few seconds can cause gaps in the time-series databases. 
> Anything that causes the self.next_chk to drift around is bad. It must 
> be a fixed interval. Shinken can still execute it a few seconds late 
> or on time, that is not a problem, but over time it should not cross 
> over a minute threshold. — Reply to this email directly or view it on 
> GitHub<https://github.com/naparuba/shinken/pull/629/files#r1784284>.
>
> —
> Reply to this email directly or view it on GitHub 
> <https://github.com/naparuba/shinken/pull/629/files#r1784683>.
>
I agree that the "next check = last planned check is 95% of the fix. The 
issue is, if the check is scheduled in the last few seconds. This will 
happen, not at initial check due to my magic lines. But when checks are 
forced over time. The 49 second thing is a corner case, but still important.

I will provide more feedback tonight and another fix, I broke something 
in my last commit. (Middle of the night for you. ;-)

Cheers,

xkilian