How long do you think this code will run?
Tags: programming, tutorial
Published: 2020-11-17 23:58:21

---
title: How long do you think this code will run?
published: true
description: 
tags: #programming #tutorial
canonical_url: https://www.jitbit.com/alexblog/292-how-long-do-you-think-this-code-will-run/
cover_image: https://i.imgur.com/289tUjL.jpg?1
---

We have just spent the last 24 hours fixing a nasty bug in production.

But first, here's a tricky question for you: <b>for how long do you think this code will run?</b>

```
//I'm trying to be language agnostic here
for (i=0; i<1000000; i++)
   thread.sleep(1);
```

<p>It would seem that the answer is obvious - a million milliseconds, which is about 15 minutes.</p>

<p><b>WRONG.</b></p>

<p>The correct answer is - <i>more than 4 hours</i>.</p>

<p>Wait, what?! Bare with me ;-)</p>

<h2>The nasty production bug story</h2>

We have a background worker on the backend - a huge `while` loop that runs through an array of millions of elements and does all kinds of in-memory checks and manipulations on them.

<p>But we don’t want the CPU to get stuck at 100% during this loop and choke the server, do we? We want the server to stay alive and kicking.</p>

<p>So what does the average Joe programmer do? That's right - Joe adds a short pause into the cycle and goes home.</p>

<blockquote>if there's any game developers reading this - I can already hear you giggling and reaching out for popcorn</blockquote>

Here's the thing: "pauses" AKA `Delay()` AKA <code>Sleep()</code> in most operating systems are based on *timers*. The resolution of these timers is 12-15ms. You cannot pause for 1 millisecond - there will be at least 15.

<p>So on a large array with a million elements, we get <code>15ms * 1000000 / 1000 / 60 / 60 = 4.16</code> - more than four hours.</p>

<p>And coming back to work in the morning our Joe-the-programmer sees what? That his loop is <i>still running</i> from yesterday. The job, that used to take 7 minutes (although it kept the CPU at 100%), now takes half a day to finish. In a "relaxed" mode though.</p>

<p>Everything is broken and customers start creating the "huh?!" tickets.</p>

<blockquote>and the gamedevs reading this - laugh viciously</blockquote>

<p>Because in their game development world this happens all the time, this is called a "busy loop" or a "tight loop". And you can't rely on timers.</p>

<h2>But how do we throttle properly?</h2>

<p>1. Use multimedia timers or timers from OpenGL/DirectX (overkill) </p>

<p>2. Throttle every N-th step, not every step (inelegant and stinks)</p>

<p>3. Dump "pauses" completely and use the magic <code>Thread.Yield</code> instruction - which is a "polite" way to share resources and tell the operating system "hey, I'm still busy, but if you really need this, slow me down and let other threads do the work" (and this is the best way)</p>

<p>The 100% CPU load won't go away, but now it's not an issue - everything is fast and responsive.</p>

<p><code>Thread.Yield</code> is available in many languages:</p>

<p><b>C#:</b> <code>Thread.Yield</code></p>

<p><b>C++:</b> <code>std::this_thread::yield</code></p>

<p><b>Win32:</b> <code>SwitchToThread</code></p>

<p><b>Java:</b> <code>Thread.yield</code></p>

<p><b>Go:</b> <code>runtime.Gosched</code> (I think...)</p>

<p><b>Visual Basic:</b> <code>DoEvents</code> (kidding! ...although not really)</p>

<p><b>Python:</b> <code>time.sleep(0)</code> (on Windows it's <code>time.sleep(0.0001)</code> don't ask me why... because Python...)</p>

<p>(by the way, it's not just Python - quite a few system libraries are smart enough to translate <code>sleep(0)</code> into a <code>yield</code>, including .NET, Posix and WinApi)</p>

<p>etc. Google your favorite language.</p>

<p>The moral of the story, I guess, is that even the trivial things get super tricky <i>at scale</i>. Things that used to be nice and simple when we just launched our little SaaS now get complicated when you have thousands of companies using your stuff. But that's a nice problem to have, I guess.</p>