Find the nearest value in the list

username: Timofey_Antonenko
cooked: <p>Hi,</p>
<p>I’d like to find an index of the biggest value in the array that smaller than my provided number.<br>
In this array, I will have more than 2 values.<br>
Example:</p>
<pre><code class="lang-auto">val = 0.25
arr = [0, 0.1, 0.2, 0.4, 0.42, 0.5, 0.56, 1]
result = my_func(arr, val)
</code></pre>
<p>So the first smaller value is 0.2 and</p>
<pre><code class="lang-auto">result = 2
</code></pre>
<p>Right now I’m doing it with python implementation of binary search. But I think there is some faster approach for PyTorch.<br>
Because it’s a huge bottleneck for me.<br>
Maybe it’s similar to MaxPool implementation, but I don’t really know.</p>
<p>Thank you,<br>
Tim</p>

username: ptrblck
cooked: <p>Wouldn’t <code>0</code> be the first smaller number in the array?<br>
How are you calculating the method at the moment?</p>

username: Timofey_Antonenko
cooked: <p>Yes, thanks - I’ve changed the description of the issue.<br>
I need an index of the biggest value in the array that is smaller than my provided value.<br>
Right now I’m using binary search.<br>
I’m using this Python module from the standard library:<br>
<a href="https://docs.python.org/3/library/bisect.html" class="onebox" target="_blank" rel="nofollow noopener">https://docs.python.org/3/library/bisect.html</a><br>
for binary search. Method - <code>bisect.</code> <code>bisect_left</code>.</p>
<p>Example of usage:</p>
<pre><code class="lang-auto">from bisect import bisect_left
nuclears = [0, 0.1, 0.15, 0.2, 0.3, 0.33, 0.5, 0.51, 0.8, 1]
x = 0.505
idx = bisect_left(nuclears, x)
idx
7
</code></pre>
<p>But I need to do this operation a lot of times and it slows me down.</p>

username: Zhaoyi-Yan
cooked: <p>As the order of array is already ascending, so you can try this:</p>
<pre><code class="lang-python">import torch
val = 0.505
nuclears = [0, 0.1, 0.15, 0.2, 0.3, 0.33, 0.5, 0.51, 0.8, 1]
mat = torch.Tensor(nuclears)
print(mat &gt; val)

mat_tmp = torch.min((mat &gt; val), 0)

print(mat_tmp[1].item() + 1)
</code></pre>

username: Timofey_Antonenko
cooked: <p>Nice approach.<br>
I wrote a small benchmark:</p>
<pre><code class="lang-auto">def comparing_idx_search_methods():
    import random
    import time

    def bisect_search(nuclears, x):
        idx = bisect_left(nuclears, x)
        return idx

    def pytorch_search(nuclears, x):
        mat_tmp = torch.min((nuclears &gt; x), 0)
        return mat_tmp[1].item() + 1

    number_of_iterations = 100000
    nuclears = [0, 0.1, 0.15, 0.2, 0.3, 0.33, 0.5, 0.51, 0.8, 1]

    x_s = [random.random() for _ in range(number_of_iterations)]

    start = time.time()
    for val in x_s:
        idx = bisect_search(nuclears, val)
    end = time.time()
    execution_time = end - start
    print("bisect: {} seconds".format(round(execution_time, 2)))

    start = time.time()
    nuclears = torch.Tensor(nuclears)
    for val in x_s:
        idx = pytorch_search(nuclears, val)
    end = time.time()
    execution_time = end - start
    print("pytorch: {} seconds".format(round(execution_time, 2)))
</code></pre>
<p>and basic output is:</p>
<pre><code class="lang-auto">bisect: 0.06 seconds
pytorch: 2.17 seconds
</code></pre>
<p>Of course - the reason is that bisect uses binary search and does not run through all elements.<br>
But it was a nice try - I thought maybe pytorch uses something under the hood and it would be faster.</p>
<p>I think maybe the solution has to be connected with how maxpool works but I don’t know where to dig next.</p>

username: Zhaoyi-Yan
cooked: <pre data-code-wrap="python"><code class="lang-python">from bisect import bisect_left
import torch
import torch.nn.functional as F

def comparing_idx_search_methods():
    import random
    import time

    def bisect_search(nuclears, x):
        idx = bisect_left(nuclears, x)
        return idx

    def pytorch_search(nuclears, x):
        tmp_feat = 1/ (1 + F.relu(nuclears - x))
        mat_tmp = torch.max(tmp_feat, dim=1)
        return mat_tmp[1] + 1

    number_of_iterations = 1000000
    nuclears = [0, 0.1, 0.15, 0.2, 0.3, 0.33, 0.5, 0.51, 0.8, 1]

    x_s = [random.random() for _ in range(number_of_iterations)]


    start = time.time()
    for val in x_s:
        idx = bisect_search(nuclears, val)
    
    end = time.time()
    execution_time = end - start
    print("bisect: {} seconds".format(round(execution_time, 2)))

    start = time.time()
    nuclears = torch.Tensor(nuclears).view(1, -1)
    x_s_th = torch.Tensor(x_s).view(-1, 1)

    idx_th = pytorch_search(nuclears, x_s_th)

    # for val in x_s:
    #     idx = pytorch_search(nuclears, val)
    end = time.time()
    execution_time = end - start
    print("pytorch: {} seconds".format(round(execution_time, 2)))

comparing_idx_search_methods()
</code></pre>
<p>Just a few modifications, making it computing-parallel. Now, it surpasses normal <code>for-loop</code> in numpy.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/1/9193a472c57097c7baf9f8919be4f8e7c9d4de82.jpeg" alt="{3D6CCE8D-663B-4613-970C-0D1BF0C7571A}_20200322224438" data-base62-sha1="kLPyrwvaWgKeSg65EDAlsC7zGEi" width="227" height="47"></p>