Dividing two floats

username: ChauhanT
cooked: <p>Hi,</p>
<p>I can’t seem to understand how one controls the precision when dividing two floats. For instance,</p>
<pre><code class="lang-auto">x = torch.tensor(10e-3)
y = torch.tensor(1e-3)
qt = x/y
qt_int = int(x/y)
print(f"{qt}")
print(f"{qt_int}")
</code></pre>
<p>Gives 9.999999046325684 and 9. Is there a way to make sure that it evaluates to 10? Perhaps I will need to specify the tolerance somewhere? Any tips/suggestions/help in understanding this would be really awesome.</p>
<p>Thanks very much!</p>

username: ptrblck
cooked: <p>No, there isn’t an option to increase the precision besides using a wider <code>dtype</code> (e.g. <code>float64</code>) which would yield an increased precision in the result.</p>
<p>You are currently experiencing the limited floating point precision as described <a href="https://en.wikipedia.org/wiki/Single-precision_floating-point_format">here</a>.</p>
<p>The input values are not exactly representable as seen here:</p>
<pre><code class="lang-python">torch.set_printoptions(precision=20)

torch.tensor(1e-3)
# tensor(0.00100000004749745131)
torch.tensor(10e-3)
# tensor(0.00999999977648258209)
</code></pre>
<p>which explains the small expected error.</p>

username: ChauhanT
cooked: <p>Thank you. I have implemented a workaround using rounding-off. But I was wondering, how do more experienced programmers deal with this problem ?</p>
<p>A better description of the problem: I have a discrete time-series x[n] with a resolution of dt (i.e., consecutive measurements x[k] and x[k+1] are sampled time dt apart). I want to bin this time-series using a user-provided interval DT. The precision-issue arose when I tried to determine the bin-size in number of samples using <code>torch.floor(DT/dt)</code>, and used <code>DT = 10e-3</code> and <code>dt=1e-3</code>.</p>
<p>Any tips would be awesome. This binning will be fundamental to my code, so I want to make sure I’ve got it as well-implemented as possible.</p>
<p>Cheers</p>

username: KFrank
cooked: <p>Hi Chauhan!</p>
<aside class="quote no-group" data-username="ChauhanT" data-post="3" data-topic="147952" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/f0a364/48.png" class="avatar"> ChauhanT:</div>
<blockquote>
<p>I want to bin this time-series using a user-provided interval DT. The precision-issue arose when I tried to determine the bin-size in number of samples using <code>torch.floor(DT/dt)</code>, and used <code>DT = 10e-3</code> and <code>dt=1e-3</code>.</p>
</blockquote>
</aside>
<p>The best approach – if you can manage it – would be to work with<br>
exact integers, rather than potentially-inexact floating-point numbers.<br>
So, for example, express <code>DT</code> as an integer, in units of <code>dt</code>.  If you need,<br>
at some point, floating-point numbers, temporarily convert your integers<br>
to floating-point numbers when you use them as such.</p>
<p>If your user insists on providing you <code>DT</code> as a floating-point number<br>
(that is expected to be an integer multiple of <code>dt</code>), then use <code>round()</code><br>
(rather than<code>floor()</code> or <code>int()</code>) to obtain the  integer.  You can optionally<br>
validate that the result is close enough to being an integer not to be<br>
a mistake:</p>
<pre data-code-wrap="python"><code class="lang-python">(torch.round (DT / dt) / (DT / dt) - 1).abs() &lt; tol
</code></pre>
<p>for some suitably chosen tolerance, <code>tol</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Andrei_Cristea
cooked: <p>Why use torch floats for this particular operation? Regular Python types work fine:</p>
<pre><code class="lang-auto">In [1]: DT = 10e-3

In [2]: dt = 1e-3

In [3]: DT/dt
Out[3]: 10.0

In [4]: int(DT/dt)
Out[4]: 10
</code></pre>
<p>Normally you should work with torch objects when it’s important for speed / efficiency sake, but this seems like a one-off operation that you can do once (or a relatively low number of times) and move on with your life without much drag to overall program performance.</p>
<p>Cheers,<br>
Andrei</p>

username: KFrank
cooked: <p>Hi Andrei!</p>
<aside class="quote no-group quote-modified" data-username="Andrei_Cristea" data-post="5" data-topic="147952" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andrei_cristea/48/47843_2.png" class="avatar"> Andrei_Cristea:</div>
<blockquote>
<p>Regular Python types work fine:</p>
<pre><code class="lang-auto">In [4]: int(DT/dt)
Out[4]: 10
</code></pre>
</blockquote>
</aside>
<p>Using python types doesn’t fix (or otherwise address) the issue of<br>
round-off error.  By default, pytorch uses single-precision (<code>float32</code>)<br>
floating-point numbers, while python uses double-precision (<code>float64</code>).<br>
This changes for which specific values this specific symptom shows<br>
up.  It doesn’t make the core problem go away – it just moves it around<br>
from one place to another:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import sys
&gt;&gt;&gt; sys.version
'3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]'
&gt;&gt;&gt; DT = 10e-17
&gt;&gt;&gt; dt = 1e-17
&gt;&gt;&gt; DT / dt
9.999999999999998
&gt;&gt;&gt; int (DT / dt)
9
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

username: Andrei_Cristea
cooked: <p>Great point!</p>
<p>This is hacky, but you could always do something like this assuming you have some hard floor on the minimum resolution. The example below assumes that hard floor happens to be 1e-33.</p>
<pre><code class="lang-auto">x = 10e-33
y = 1e-33

(x * 1e+17) / (y * 1e+17)
Out: 10.0

int((x * 1e+17) / (y * 1e+17))
Out: 10
</code></pre>
<p>In practice, it’s hard to believe you’re not going to have <em>some</em> lower bound on the resolution interval. 1e-3 is easy to cross in many practical cases (as absurd as it is to make that statement in general rather than with reference to a particular domain, it is nonetheless plausible), 1e-33 less so.</p>

username: KFrank
cooked: <p>Hi Andrei!</p>
<aside class="quote no-group" data-username="Andrei_Cristea" data-post="7" data-topic="147952" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andrei_cristea/48/47843_2.png" class="avatar"> Andrei_Cristea:</div>
<blockquote>
<p>The example below assumes that hard floor happens to be 1e-33.</p>
<pre><code class="lang-auto">x = 10e-33
y = 1e-33

(x * 1e+17) / (y * 1e+17)
Out: 10.0

int((x * 1e+17) / (y * 1e+17))
Out: 10
</code></pre>
</blockquote>
</aside>
<p>The issue is not whether there is some (small) floor to the resolution<br>
interval, <code>y</code> (called <code>dt</code> in the posts above).</p>
<p>Instead, the question is whether an “integer” multiple of <code>y</code>, including<br>
its round-off error, falls a little bit below or a little bit above (or exactly<br>
on) the true integer multiple of <code>y</code>.  You can fall a little below the true<br>
integer multiple – and hence get the wrong result when converting<br>
(truncating) to <code>int</code> – even for a “large” resolution interval such as <code>1e-1</code>:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import sys
&gt;&gt;&gt; sys.version
'3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]'
&gt;&gt;&gt; x = 7e-1
&gt;&gt;&gt; y = 1e-1
&gt;&gt;&gt; x / y
6.999999999999999
&gt;&gt;&gt; int (x / y)
6
&gt;&gt;&gt; (x * 1e+17) / (y * 1e+17)
6.999999999999999
&gt;&gt;&gt; int ((x * 1e+17) / (y * 1e+17))
6
</code></pre>
<p>For the use case that I believe Chauhan envisions it should suffice to<br>
<code>round()</code> to the nearest integer, rather than truncating to <code>int()</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

username: Andrei_Cristea
cooked: <p>Wow!</p>
<pre><code class="lang-auto">0.7 / 0.1 == 7.0
Out: False
</code></pre>
<p>I guess they don’t make 'em like they used to…</p>

username: ChauhanT
cooked: <p>Hi,</p>
<p>Thank you for this very interesting discussion! My problem gets compounded by the constraints of the mapping. Ideally, I’d like to use a <code>floor</code> operation to have a conservative bin-size. But the rounding-off makes the problem tricky. Suppose the function is <code>f(Dt,dt)</code>, I’d like:</p>
<pre><code class="lang-auto">Dt,dt = 10.0,1.0
f(Dt,dt) = 10

Dt,dt = 9.9,1.0
f(Dt,dt) = 9
</code></pre>
<p>If <code>10.0/1.0</code> evaluated to <code>10.0</code>, I could accomplish this by a simple <code>floor</code>. But alas I have to resort to other tricks. I have come up with:</p>
<pre><code class="lang-auto">def compute_n_bins(Dt,dt,relTol=1e-3):
  qt = Dt/dt
  n_bins = torch.floor(qt)

  # [qt] can only approach 1 from the left
  if 1 - torch.frac(qt) &lt; relTol:
    n_bins += 1

  return int(n_bins)
</code></pre>
<p>I think primitive float types for computational framework should implement integer division checks (python native floats do this? Maybe? <span class="hashtag">#Noob</span>). But I guess we work with what we got! Plus, such checks will probably add a speed penalty in cases where there is no need for one.</p>
<p>Please let me know if you think there is something blatantly silly happening in my <code>compute_n_bins</code>, or if you know of a simpler/more direct way of doing this. I recognise that <code>torch.frac</code> is probably an expensive operation, but I am hoping to run <code>compute_n_bins</code> less than 10 times in the whole computation.</p>
<p>Cheers!</p>