How to reduce latency of transferring tensors from CPU to GPU

incerto: <p>Hi PyTorch team,</p>
<p>I am looking for the fastest way to transfer tensors from CPU to GPU. According to <a href="https://developer.nvidia.com/blog/how-optimize-data-transfers-cuda-cc" rel="noopener nofollow ugc">the blog</a> from Nvidia, data transfer is faster on pinned memory. So I ran a simple experiment using the following code on A100.</p>
<pre><code class="lang-python">import time
import torch
import pandas as pd
from tqdm import tqdm


def get_performance_metrics(x):
    shape = tuple(x.size())

    time_start = time.perf_counter()
    x = x.pin_memory()
    torch.cuda.synchronize()
    elapsed_pin = time.perf_counter() - time_start

    time_start = time.perf_counter()
    x = x.to("cuda:0")
    torch.cuda.synchronize()
    elapsed_cuda = time.perf_counter() - time_start

    return {
        "shape": shape,
        "pin-memory(ms)": elapsed_pin * 1000,
        "pinned-&gt;cuda(ms)": elapsed_cuda * 1000,
    }

df = []
for seq in tqdm(range(1, 1050, 5)):
    x = torch.rand((24, 2, 1, seq, 4096)).type(torch.float16)
    m = get_performance_metrics(x)
    del x
    torch.cuda.empty_cache()

    y = torch.rand((24, 2, 1, seq, 4096)).type(torch.float16)
    time_start = time.perf_counter()
    y.to("cuda")
    torch.cuda.synchronize()
    elapsed_cuda = time.perf_counter() - time_start
    m["direct to cuda(ms)"] = elapsed_cuda * 1000

    del y
    torch.cuda.empty_cache()

    df.append(m)

df = pd.DataFrame(df).round(2)
df.to_excel("tensor_copy_measurement.xlsx", index=False)

</code></pre>
<p>In general, there is a trend that pinning the tensor first will produce a lower transfer latency. But this does not hold for all data points. Also there are two weird phenomena</p>
<ol>
<li>Transferring a tensor of shape (24, 2, 1, 1026, 4096) is faster than (24, 2, 1, 966, 4096)</li>
<li>There are some outliers which take significantly more time.</li>
</ol>
<div class="md-table">
<table>
<thead>
<tr>
<th style="text-align:right"></th>
<th style="text-align:left">shape</th>
<th style="text-align:right">pin-memory(ms)</th>
<th style="text-align:right">pinned-&gt;cuda(ms)</th>
<th style="text-align:right">direct to cuda(ms)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:right">0</td>
<td style="text-align:left">(24, 2, 1, 1, 4096)</td>
<td style="text-align:right">1245.59</td>
<td style="text-align:right">0.33</td>
<td style="text-align:right">0.24</td>
</tr>
<tr>
<td style="text-align:right">1</td>
<td style="text-align:left">(24, 2, 1, 6, 4096)</td>
<td style="text-align:right">1.47</td>
<td style="text-align:right">0.3</td>
<td style="text-align:right">0.42</td>
</tr>
<tr>
<td style="text-align:right">2</td>
<td style="text-align:left">(24, 2, 1, 11, 4096)</td>
<td style="text-align:right">2.45</td>
<td style="text-align:right">0.42</td>
<td style="text-align:right">0.53</td>
</tr>
<tr>
<td style="text-align:right">3</td>
<td style="text-align:left">(24, 2, 1, 16, 4096)</td>
<td style="text-align:right">0.22</td>
<td style="text-align:right">0.54</td>
<td style="text-align:right">0.66</td>
</tr>
<tr>
<td style="text-align:right">4</td>
<td style="text-align:left">(24, 2, 1, 21, 4096)</td>
<td style="text-align:right">0.19</td>
<td style="text-align:right">0.59</td>
<td style="text-align:right">0.81</td>
</tr>
<tr>
<td style="text-align:right">5</td>
<td style="text-align:left">(24, 2, 1, 26, 4096)</td>
<td style="text-align:right">4.83</td>
<td style="text-align:right">0.73</td>
<td style="text-align:right">1</td>
</tr>
<tr>
<td style="text-align:right">6</td>
<td style="text-align:left">(24, 2, 1, 31, 4096)</td>
<td style="text-align:right">0.31</td>
<td style="text-align:right">0.86</td>
<td style="text-align:right">1.1</td>
</tr>
<tr>
<td style="text-align:right">7</td>
<td style="text-align:left">(24, 2, 1, 36, 4096)</td>
<td style="text-align:right">0.38</td>
<td style="text-align:right">0.98</td>
<td style="text-align:right">1.27</td>
</tr>
<tr>
<td style="text-align:right">8</td>
<td style="text-align:left">(24, 2, 1, 41, 4096)</td>
<td style="text-align:right">0.45</td>
<td style="text-align:right">1.13</td>
<td style="text-align:right">1.46</td>
</tr>
<tr>
<td style="text-align:right">9</td>
<td style="text-align:left">(24, 2, 1, 46, 4096)</td>
<td style="text-align:right">8.2</td>
<td style="text-align:right">1.19</td>
<td style="text-align:right">1.62</td>
</tr>
<tr>
<td style="text-align:right">10</td>
<td style="text-align:left">(24, 2, 1, 51, 4096)</td>
<td style="text-align:right">0.56</td>
<td style="text-align:right">1.31</td>
<td style="text-align:right">1.86</td>
</tr>
<tr>
<td style="text-align:right">11</td>
<td style="text-align:left">(24, 2, 1, 56, 4096)</td>
<td style="text-align:right">0.64</td>
<td style="text-align:right">1.46</td>
<td style="text-align:right">2.11</td>
</tr>
<tr>
<td style="text-align:right">12</td>
<td style="text-align:left">(24, 2, 1, 61, 4096)</td>
<td style="text-align:right">0.71</td>
<td style="text-align:right">1.54</td>
<td style="text-align:right">2.36</td>
</tr>
<tr>
<td style="text-align:right">13</td>
<td style="text-align:left">(24, 2, 1, 66, 4096)</td>
<td style="text-align:right">1.04</td>
<td style="text-align:right">1.67</td>
<td style="text-align:right">91.19</td>
</tr>
<tr>
<td style="text-align:right">14</td>
<td style="text-align:left">(24, 2, 1, 71, 4096)</td>
<td style="text-align:right">1.09</td>
<td style="text-align:right">1.74</td>
<td style="text-align:right">2.82</td>
</tr>
<tr>
<td style="text-align:right">15</td>
<td style="text-align:left">(24, 2, 1, 76, 4096)</td>
<td style="text-align:right">1.15</td>
<td style="text-align:right">1.93</td>
<td style="text-align:right">2.99</td>
</tr>
<tr>
<td style="text-align:right">16</td>
<td style="text-align:left">(24, 2, 1, 81, 4096)</td>
<td style="text-align:right">1.22</td>
<td style="text-align:right">1.97</td>
<td style="text-align:right">3.2</td>
</tr>
<tr>
<td style="text-align:right">17</td>
<td style="text-align:left">(24, 2, 1, 86, 4096)</td>
<td style="text-align:right">17.01</td>
<td style="text-align:right">2.04</td>
<td style="text-align:right">3.53</td>
</tr>
<tr>
<td style="text-align:right">18</td>
<td style="text-align:left">(24, 2, 1, 91, 4096)</td>
<td style="text-align:right">1.06</td>
<td style="text-align:right">2.16</td>
<td style="text-align:right">3.67</td>
</tr>
<tr>
<td style="text-align:right">19</td>
<td style="text-align:left">(24, 2, 1, 96, 4096)</td>
<td style="text-align:right">1.16</td>
<td style="text-align:right">120.73</td>
<td style="text-align:right">3.93</td>
</tr>
<tr>
<td style="text-align:right">20</td>
<td style="text-align:left">(24, 2, 1, 101, 4096)</td>
<td style="text-align:right">1.16</td>
<td style="text-align:right">2.42</td>
<td style="text-align:right">4.08</td>
</tr>
<tr>
<td style="text-align:right">21</td>
<td style="text-align:left">(24, 2, 1, 106, 4096)</td>
<td style="text-align:right">1.22</td>
<td style="text-align:right">2.5</td>
<td style="text-align:right">4.26</td>
</tr>
<tr>
<td style="text-align:right">22</td>
<td style="text-align:left">(24, 2, 1, 111, 4096)</td>
<td style="text-align:right">1.28</td>
<td style="text-align:right">2.6</td>
<td style="text-align:right">4.45</td>
</tr>
<tr>
<td style="text-align:right">23</td>
<td style="text-align:left">(24, 2, 1, 116, 4096)</td>
<td style="text-align:right">1.35</td>
<td style="text-align:right">2.68</td>
<td style="text-align:right">64.01</td>
</tr>
<tr>
<td style="text-align:right">24</td>
<td style="text-align:left">(24, 2, 1, 121, 4096)</td>
<td style="text-align:right">1.4</td>
<td style="text-align:right">2.78</td>
<td style="text-align:right">4.88</td>
</tr>
<tr>
<td style="text-align:right">25</td>
<td style="text-align:left">(24, 2, 1, 126, 4096)</td>
<td style="text-align:right">1.43</td>
<td style="text-align:right">2.92</td>
<td style="text-align:right">5.13</td>
</tr>
<tr>
<td style="text-align:right">26</td>
<td style="text-align:left">(24, 2, 1, 131, 4096)</td>
<td style="text-align:right">1.5</td>
<td style="text-align:right">3</td>
<td style="text-align:right">5.29</td>
</tr>
<tr>
<td style="text-align:right">27</td>
<td style="text-align:left">(24, 2, 1, 136, 4096)</td>
<td style="text-align:right">1.56</td>
<td style="text-align:right">142.71</td>
<td style="text-align:right">5.46</td>
</tr>
<tr>
<td style="text-align:right">28</td>
<td style="text-align:left">(24, 2, 1, 141, 4096)</td>
<td style="text-align:right">1.66</td>
<td style="text-align:right">3.23</td>
<td style="text-align:right">5.67</td>
</tr>
<tr>
<td style="text-align:right">29</td>
<td style="text-align:left">(24, 2, 1, 146, 4096)</td>
<td style="text-align:right">1.68</td>
<td style="text-align:right">3.37</td>
<td style="text-align:right">5.86</td>
</tr>
<tr>
<td style="text-align:right">30</td>
<td style="text-align:left">(24, 2, 1, 151, 4096)</td>
<td style="text-align:right">1.72</td>
<td style="text-align:right">3.39</td>
<td style="text-align:right">11.65</td>
</tr>
<tr>
<td style="text-align:right">31</td>
<td style="text-align:left">(24, 2, 1, 156, 4096)</td>
<td style="text-align:right">1.75</td>
<td style="text-align:right">3.5</td>
<td style="text-align:right">6.22</td>
</tr>
<tr>
<td style="text-align:right">32</td>
<td style="text-align:left">(24, 2, 1, 161, 4096)</td>
<td style="text-align:right">1.79</td>
<td style="text-align:right">3.63</td>
<td style="text-align:right">6.46</td>
</tr>
<tr>
<td style="text-align:right">33</td>
<td style="text-align:left">(24, 2, 1, 166, 4096)</td>
<td style="text-align:right">1.87</td>
<td style="text-align:right">3.72</td>
<td style="text-align:right">63.62</td>
</tr>
<tr>
<td style="text-align:right">34</td>
<td style="text-align:left">(24, 2, 1, 171, 4096)</td>
<td style="text-align:right">35.3</td>
<td style="text-align:right">3.78</td>
<td style="text-align:right">6.86</td>
</tr>
<tr>
<td style="text-align:right">35</td>
<td style="text-align:left">(24, 2, 1, 176, 4096)</td>
<td style="text-align:right">1.98</td>
<td style="text-align:right">3.92</td>
<td style="text-align:right">7.02</td>
</tr>
<tr>
<td style="text-align:right">36</td>
<td style="text-align:left">(24, 2, 1, 181, 4096)</td>
<td style="text-align:right">2.7</td>
<td style="text-align:right">128.2</td>
<td style="text-align:right">7.37</td>
</tr>
<tr>
<td style="text-align:right">37</td>
<td style="text-align:left">(24, 2, 1, 186, 4096)</td>
<td style="text-align:right">2.79</td>
<td style="text-align:right">4.1</td>
<td style="text-align:right">7.38</td>
</tr>
<tr>
<td style="text-align:right">38</td>
<td style="text-align:left">(24, 2, 1, 191, 4096)</td>
<td style="text-align:right">2.86</td>
<td style="text-align:right">4.24</td>
<td style="text-align:right">92.4</td>
</tr>
<tr>
<td style="text-align:right">39</td>
<td style="text-align:left">(24, 2, 1, 196, 4096)</td>
<td style="text-align:right">2.93</td>
<td style="text-align:right">4.26</td>
<td style="text-align:right">7.77</td>
</tr>
<tr>
<td style="text-align:right">40</td>
<td style="text-align:left">(24, 2, 1, 201, 4096)</td>
<td style="text-align:right">3.01</td>
<td style="text-align:right">4.39</td>
<td style="text-align:right">8.01</td>
</tr>
<tr>
<td style="text-align:right">41</td>
<td style="text-align:left">(24, 2, 1, 206, 4096)</td>
<td style="text-align:right">3.08</td>
<td style="text-align:right">11.48</td>
<td style="text-align:right">8.2</td>
</tr>
<tr>
<td style="text-align:right">42</td>
<td style="text-align:left">(24, 2, 1, 211, 4096)</td>
<td style="text-align:right">3.14</td>
<td style="text-align:right">4.61</td>
<td style="text-align:right">8.46</td>
</tr>
<tr>
<td style="text-align:right">43</td>
<td style="text-align:left">(24, 2, 1, 216, 4096)</td>
<td style="text-align:right">3.21</td>
<td style="text-align:right">4.68</td>
<td style="text-align:right">8.62</td>
</tr>
<tr>
<td style="text-align:right">44</td>
<td style="text-align:left">(24, 2, 1, 221, 4096)</td>
<td style="text-align:right">3.29</td>
<td style="text-align:right">4.8</td>
<td style="text-align:right">8.77</td>
</tr>
<tr>
<td style="text-align:right">45</td>
<td style="text-align:left">(24, 2, 1, 226, 4096)</td>
<td style="text-align:right">3.36</td>
<td style="text-align:right">4.88</td>
<td style="text-align:right">87.67</td>
</tr>
<tr>
<td style="text-align:right">46</td>
<td style="text-align:left">(24, 2, 1, 231, 4096)</td>
<td style="text-align:right">3.43</td>
<td style="text-align:right">4.96</td>
<td style="text-align:right">9.14</td>
</tr>
<tr>
<td style="text-align:right">47</td>
<td style="text-align:left">(24, 2, 1, 236, 4096)</td>
<td style="text-align:right">3.49</td>
<td style="text-align:right">5.06</td>
<td style="text-align:right">100.49</td>
</tr>
<tr>
<td style="text-align:right">48</td>
<td style="text-align:left">(24, 2, 1, 241, 4096)</td>
<td style="text-align:right">3.52</td>
<td style="text-align:right">5.26</td>
<td style="text-align:right">9.56</td>
</tr>
<tr>
<td style="text-align:right">49</td>
<td style="text-align:left">(24, 2, 1, 246, 4096)</td>
<td style="text-align:right">3.58</td>
<td style="text-align:right">5.35</td>
<td style="text-align:right">54.47</td>
</tr>
<tr>
<td style="text-align:right">50</td>
<td style="text-align:left">(24, 2, 1, 251, 4096)</td>
<td style="text-align:right">3.64</td>
<td style="text-align:right">5.43</td>
<td style="text-align:right">9.98</td>
</tr>
<tr>
<td style="text-align:right">51</td>
<td style="text-align:left">(24, 2, 1, 256, 4096)</td>
<td style="text-align:right">3.71</td>
<td style="text-align:right">5.53</td>
<td style="text-align:right">10.14</td>
</tr>
<tr>
<td style="text-align:right">52</td>
<td style="text-align:left">(24, 2, 1, 261, 4096)</td>
<td style="text-align:right">3.77</td>
<td style="text-align:right">5.62</td>
<td style="text-align:right">10.37</td>
</tr>
<tr>
<td style="text-align:right">53</td>
<td style="text-align:left">(24, 2, 1, 266, 4096)</td>
<td style="text-align:right">3.86</td>
<td style="text-align:right">5.74</td>
<td style="text-align:right">10.53</td>
</tr>
<tr>
<td style="text-align:right">54</td>
<td style="text-align:left">(24, 2, 1, 271, 4096)</td>
<td style="text-align:right">3.94</td>
<td style="text-align:right">5.81</td>
<td style="text-align:right">10.72</td>
</tr>
<tr>
<td style="text-align:right">55</td>
<td style="text-align:left">(24, 2, 1, 276, 4096)</td>
<td style="text-align:right">4.02</td>
<td style="text-align:right">5.93</td>
<td style="text-align:right">10.9</td>
</tr>
<tr>
<td style="text-align:right">56</td>
<td style="text-align:left">(24, 2, 1, 281, 4096)</td>
<td style="text-align:right">4.09</td>
<td style="text-align:right">6.01</td>
<td style="text-align:right">11.16</td>
</tr>
<tr>
<td style="text-align:right">57</td>
<td style="text-align:left">(24, 2, 1, 286, 4096)</td>
<td style="text-align:right">4.14</td>
<td style="text-align:right">38.6</td>
<td style="text-align:right">11.33</td>
</tr>
<tr>
<td style="text-align:right">58</td>
<td style="text-align:left">(24, 2, 1, 291, 4096)</td>
<td style="text-align:right">4.21</td>
<td style="text-align:right">6.29</td>
<td style="text-align:right">11.53</td>
</tr>
<tr>
<td style="text-align:right">59</td>
<td style="text-align:left">(24, 2, 1, 296, 4096)</td>
<td style="text-align:right">4.27</td>
<td style="text-align:right">6.25</td>
<td style="text-align:right">11.7</td>
</tr>
<tr>
<td style="text-align:right">60</td>
<td style="text-align:left">(24, 2, 1, 301, 4096)</td>
<td style="text-align:right">4.36</td>
<td style="text-align:right">6.46</td>
<td style="text-align:right">11.92</td>
</tr>
<tr>
<td style="text-align:right">61</td>
<td style="text-align:left">(24, 2, 1, 306, 4096)</td>
<td style="text-align:right">4.42</td>
<td style="text-align:right">6.51</td>
<td style="text-align:right">12.11</td>
</tr>
<tr>
<td style="text-align:right">62</td>
<td style="text-align:left">(24, 2, 1, 311, 4096)</td>
<td style="text-align:right">4.5</td>
<td style="text-align:right">65.78</td>
<td style="text-align:right">12.26</td>
</tr>
<tr>
<td style="text-align:right">63</td>
<td style="text-align:left">(24, 2, 1, 316, 4096)</td>
<td style="text-align:right">4.56</td>
<td style="text-align:right">6.77</td>
<td style="text-align:right">95.24</td>
</tr>
<tr>
<td style="text-align:right">64</td>
<td style="text-align:left">(24, 2, 1, 321, 4096)</td>
<td style="text-align:right">4.64</td>
<td style="text-align:right">6.82</td>
<td style="text-align:right">12.69</td>
</tr>
<tr>
<td style="text-align:right">65</td>
<td style="text-align:left">(24, 2, 1, 326, 4096)</td>
<td style="text-align:right">4.8</td>
<td style="text-align:right">53.54</td>
<td style="text-align:right">12.9</td>
</tr>
<tr>
<td style="text-align:right">66</td>
<td style="text-align:left">(24, 2, 1, 331, 4096)</td>
<td style="text-align:right">4.86</td>
<td style="text-align:right">7.02</td>
<td style="text-align:right">43.83</td>
</tr>
<tr>
<td style="text-align:right">67</td>
<td style="text-align:left">(24, 2, 1, 336, 4096)</td>
<td style="text-align:right">4.95</td>
<td style="text-align:right">7.18</td>
<td style="text-align:right">13.26</td>
</tr>
<tr>
<td style="text-align:right">68</td>
<td style="text-align:left">(24, 2, 1, 341, 4096)</td>
<td style="text-align:right">5</td>
<td style="text-align:right">7.2</td>
<td style="text-align:right">13.45</td>
</tr>
<tr>
<td style="text-align:right">69</td>
<td style="text-align:left">(24, 2, 1, 346, 4096)</td>
<td style="text-align:right">72.38</td>
<td style="text-align:right">7.37</td>
<td style="text-align:right">13.43</td>
</tr>
<tr>
<td style="text-align:right">70</td>
<td style="text-align:left">(24, 2, 1, 351, 4096)</td>
<td style="text-align:right">5.13</td>
<td style="text-align:right">7.44</td>
<td style="text-align:right">13.54</td>
</tr>
<tr>
<td style="text-align:right">71</td>
<td style="text-align:left">(24, 2, 1, 356, 4096)</td>
<td style="text-align:right">5.19</td>
<td style="text-align:right">7.55</td>
<td style="text-align:right">13.71</td>
</tr>
<tr>
<td style="text-align:right">72</td>
<td style="text-align:left">(24, 2, 1, 361, 4096)</td>
<td style="text-align:right">5.28</td>
<td style="text-align:right">7.64</td>
<td style="text-align:right">14.06</td>
</tr>
<tr>
<td style="text-align:right">73</td>
<td style="text-align:left">(24, 2, 1, 366, 4096)</td>
<td style="text-align:right">5.35</td>
<td style="text-align:right">7.69</td>
<td style="text-align:right">99.09</td>
</tr>
<tr>
<td style="text-align:right">74</td>
<td style="text-align:left">(24, 2, 1, 371, 4096)</td>
<td style="text-align:right">5.42</td>
<td style="text-align:right">7.8</td>
<td style="text-align:right">14.34</td>
</tr>
<tr>
<td style="text-align:right">75</td>
<td style="text-align:left">(24, 2, 1, 376, 4096)</td>
<td style="text-align:right">5.49</td>
<td style="text-align:right">7.9</td>
<td style="text-align:right">14.51</td>
</tr>
<tr>
<td style="text-align:right">76</td>
<td style="text-align:left">(24, 2, 1, 381, 4096)</td>
<td style="text-align:right">5.57</td>
<td style="text-align:right">8.04</td>
<td style="text-align:right">14.67</td>
</tr>
<tr>
<td style="text-align:right">77</td>
<td style="text-align:left">(24, 2, 1, 386, 4096)</td>
<td style="text-align:right">4.19</td>
<td style="text-align:right">8.21</td>
<td style="text-align:right">14.96</td>
</tr>
<tr>
<td style="text-align:right">78</td>
<td style="text-align:left">(24, 2, 1, 391, 4096)</td>
<td style="text-align:right">4.27</td>
<td style="text-align:right">8.25</td>
<td style="text-align:right">137.99</td>
</tr>
<tr>
<td style="text-align:right">79</td>
<td style="text-align:left">(24, 2, 1, 396, 4096)</td>
<td style="text-align:right">5.81</td>
<td style="text-align:right">8.36</td>
<td style="text-align:right">15.35</td>
</tr>
<tr>
<td style="text-align:right">80</td>
<td style="text-align:left">(24, 2, 1, 401, 4096)</td>
<td style="text-align:right">5.85</td>
<td style="text-align:right">8.4</td>
<td style="text-align:right">15.59</td>
</tr>
<tr>
<td style="text-align:right">81</td>
<td style="text-align:left">(24, 2, 1, 406, 4096)</td>
<td style="text-align:right">5.96</td>
<td style="text-align:right">26.22</td>
<td style="text-align:right">15.78</td>
</tr>
<tr>
<td style="text-align:right">82</td>
<td style="text-align:left">(24, 2, 1, 411, 4096)</td>
<td style="text-align:right">5.99</td>
<td style="text-align:right">8.6</td>
<td style="text-align:right">15.93</td>
</tr>
<tr>
<td style="text-align:right">83</td>
<td style="text-align:left">(24, 2, 1, 416, 4096)</td>
<td style="text-align:right">6.08</td>
<td style="text-align:right">8.72</td>
<td style="text-align:right">16.12</td>
</tr>
<tr>
<td style="text-align:right">84</td>
<td style="text-align:left">(24, 2, 1, 421, 4096)</td>
<td style="text-align:right">6.17</td>
<td style="text-align:right">8.82</td>
<td style="text-align:right">166.67</td>
</tr>
<tr>
<td style="text-align:right">85</td>
<td style="text-align:left">(24, 2, 1, 426, 4096)</td>
<td style="text-align:right">6.24</td>
<td style="text-align:right">8.94</td>
<td style="text-align:right">16.5</td>
</tr>
<tr>
<td style="text-align:right">86</td>
<td style="text-align:left">(24, 2, 1, 431, 4096)</td>
<td style="text-align:right">6.28</td>
<td style="text-align:right">9.07</td>
<td style="text-align:right">16.67</td>
</tr>
<tr>
<td style="text-align:right">87</td>
<td style="text-align:left">(24, 2, 1, 436, 4096)</td>
<td style="text-align:right">6.36</td>
<td style="text-align:right">9.17</td>
<td style="text-align:right">17.05</td>
</tr>
<tr>
<td style="text-align:right">88</td>
<td style="text-align:left">(24, 2, 1, 441, 4096)</td>
<td style="text-align:right">6.72</td>
<td style="text-align:right">9.38</td>
<td style="text-align:right">17.11</td>
</tr>
<tr>
<td style="text-align:right">89</td>
<td style="text-align:left">(24, 2, 1, 446, 4096)</td>
<td style="text-align:right">6.74</td>
<td style="text-align:right">156.34</td>
<td style="text-align:right">17.29</td>
</tr>
<tr>
<td style="text-align:right">90</td>
<td style="text-align:left">(24, 2, 1, 451, 4096)</td>
<td style="text-align:right">6.81</td>
<td style="text-align:right">140.54</td>
<td style="text-align:right">17.45</td>
</tr>
<tr>
<td style="text-align:right">91</td>
<td style="text-align:left">(24, 2, 1, 456, 4096)</td>
<td style="text-align:right">6.9</td>
<td style="text-align:right">130</td>
<td style="text-align:right">17.72</td>
</tr>
<tr>
<td style="text-align:right">92</td>
<td style="text-align:left">(24, 2, 1, 461, 4096)</td>
<td style="text-align:right">6.92</td>
<td style="text-align:right">121.27</td>
<td style="text-align:right">17.83</td>
</tr>
<tr>
<td style="text-align:right">93</td>
<td style="text-align:left">(24, 2, 1, 466, 4096)</td>
<td style="text-align:right">7.04</td>
<td style="text-align:right">110.51</td>
<td style="text-align:right">17.98</td>
</tr>
<tr>
<td style="text-align:right">94</td>
<td style="text-align:left">(24, 2, 1, 471, 4096)</td>
<td style="text-align:right">7.09</td>
<td style="text-align:right">99.92</td>
<td style="text-align:right">18.18</td>
</tr>
<tr>
<td style="text-align:right">95</td>
<td style="text-align:left">(24, 2, 1, 476, 4096)</td>
<td style="text-align:right">7.15</td>
<td style="text-align:right">87.87</td>
<td style="text-align:right">18.54</td>
</tr>
<tr>
<td style="text-align:right">96</td>
<td style="text-align:left">(24, 2, 1, 481, 4096)</td>
<td style="text-align:right">7.26</td>
<td style="text-align:right">69.33</td>
<td style="text-align:right">18.79</td>
</tr>
<tr>
<td style="text-align:right">97</td>
<td style="text-align:left">(24, 2, 1, 486, 4096)</td>
<td style="text-align:right">7.36</td>
<td style="text-align:right">58.46</td>
<td style="text-align:right">18.97</td>
</tr>
<tr>
<td style="text-align:right">98</td>
<td style="text-align:left">(24, 2, 1, 491, 4096)</td>
<td style="text-align:right">7.43</td>
<td style="text-align:right">46.41</td>
<td style="text-align:right">19.16</td>
</tr>
<tr>
<td style="text-align:right">99</td>
<td style="text-align:left">(24, 2, 1, 496, 4096)</td>
<td style="text-align:right">7.48</td>
<td style="text-align:right">39.62</td>
<td style="text-align:right">19.4</td>
</tr>
<tr>
<td style="text-align:right">100</td>
<td style="text-align:left">(24, 2, 1, 501, 4096)</td>
<td style="text-align:right">7.57</td>
<td style="text-align:right">21.8</td>
<td style="text-align:right">19.55</td>
</tr>
<tr>
<td style="text-align:right">101</td>
<td style="text-align:left">(24, 2, 1, 506, 4096)</td>
<td style="text-align:right">7.44</td>
<td style="text-align:right">22.77</td>
<td style="text-align:right">19.85</td>
</tr>
<tr>
<td style="text-align:right">102</td>
<td style="text-align:left">(24, 2, 1, 511, 4096)</td>
<td style="text-align:right">5.78</td>
<td style="text-align:right">10.54</td>
<td style="text-align:right">19.98</td>
</tr>
<tr>
<td style="text-align:right">103</td>
<td style="text-align:left">(24, 2, 1, 516, 4096)</td>
<td style="text-align:right">5.59</td>
<td style="text-align:right">10.8</td>
<td style="text-align:right">20.09</td>
</tr>
<tr>
<td style="text-align:right">104</td>
<td style="text-align:left">(24, 2, 1, 521, 4096)</td>
<td style="text-align:right">5.66</td>
<td style="text-align:right">10.83</td>
<td style="text-align:right">20.42</td>
</tr>
<tr>
<td style="text-align:right">105</td>
<td style="text-align:left">(24, 2, 1, 526, 4096)</td>
<td style="text-align:right">5.73</td>
<td style="text-align:right">10.95</td>
<td style="text-align:right">20.54</td>
</tr>
<tr>
<td style="text-align:right">106</td>
<td style="text-align:left">(24, 2, 1, 531, 4096)</td>
<td style="text-align:right">5.75</td>
<td style="text-align:right">11.03</td>
<td style="text-align:right">20.71</td>
</tr>
<tr>
<td style="text-align:right">107</td>
<td style="text-align:left">(24, 2, 1, 536, 4096)</td>
<td style="text-align:right">5.83</td>
<td style="text-align:right">11.17</td>
<td style="text-align:right">21.24</td>
</tr>
<tr>
<td style="text-align:right">108</td>
<td style="text-align:left">(24, 2, 1, 541, 4096)</td>
<td style="text-align:right">5.87</td>
<td style="text-align:right">11.23</td>
<td style="text-align:right">21.18</td>
</tr>
<tr>
<td style="text-align:right">109</td>
<td style="text-align:left">(24, 2, 1, 546, 4096)</td>
<td style="text-align:right">5.91</td>
<td style="text-align:right">11.34</td>
<td style="text-align:right">21.24</td>
</tr>
<tr>
<td style="text-align:right">110</td>
<td style="text-align:left">(24, 2, 1, 551, 4096)</td>
<td style="text-align:right">5.98</td>
<td style="text-align:right">11.47</td>
<td style="text-align:right">138.33</td>
</tr>
<tr>
<td style="text-align:right">111</td>
<td style="text-align:left">(24, 2, 1, 556, 4096)</td>
<td style="text-align:right">8.44</td>
<td style="text-align:right">11.57</td>
<td style="text-align:right">21.63</td>
</tr>
<tr>
<td style="text-align:right">112</td>
<td style="text-align:left">(24, 2, 1, 561, 4096)</td>
<td style="text-align:right">8.48</td>
<td style="text-align:right">11.68</td>
<td style="text-align:right">21.86</td>
</tr>
<tr>
<td style="text-align:right">113</td>
<td style="text-align:left">(24, 2, 1, 566, 4096)</td>
<td style="text-align:right">8.57</td>
<td style="text-align:right">11.77</td>
<td style="text-align:right">22.08</td>
</tr>
<tr>
<td style="text-align:right">114</td>
<td style="text-align:left">(24, 2, 1, 571, 4096)</td>
<td style="text-align:right">8.64</td>
<td style="text-align:right">95.79</td>
<td style="text-align:right">22.19</td>
</tr>
<tr>
<td style="text-align:right">115</td>
<td style="text-align:left">(24, 2, 1, 576, 4096)</td>
<td style="text-align:right">8.73</td>
<td style="text-align:right">11.9</td>
<td style="text-align:right">22.39</td>
</tr>
<tr>
<td style="text-align:right">116</td>
<td style="text-align:left">(24, 2, 1, 581, 4096)</td>
<td style="text-align:right">8.78</td>
<td style="text-align:right">12.06</td>
<td style="text-align:right">22.68</td>
</tr>
<tr>
<td style="text-align:right">117</td>
<td style="text-align:left">(24, 2, 1, 586, 4096)</td>
<td style="text-align:right">8.88</td>
<td style="text-align:right">12.15</td>
<td style="text-align:right">22.82</td>
</tr>
<tr>
<td style="text-align:right">118</td>
<td style="text-align:left">(24, 2, 1, 591, 4096)</td>
<td style="text-align:right">8.95</td>
<td style="text-align:right">12.29</td>
<td style="text-align:right">23</td>
</tr>
<tr>
<td style="text-align:right">119</td>
<td style="text-align:left">(24, 2, 1, 596, 4096)</td>
<td style="text-align:right">9.01</td>
<td style="text-align:right">130.26</td>
<td style="text-align:right">23.32</td>
</tr>
<tr>
<td style="text-align:right">120</td>
<td style="text-align:left">(24, 2, 1, 601, 4096)</td>
<td style="text-align:right">9.09</td>
<td style="text-align:right">12.38</td>
<td style="text-align:right">23.5</td>
</tr>
<tr>
<td style="text-align:right">121</td>
<td style="text-align:left">(24, 2, 1, 606, 4096)</td>
<td style="text-align:right">9.18</td>
<td style="text-align:right">12.51</td>
<td style="text-align:right">44.14</td>
</tr>
<tr>
<td style="text-align:right">122</td>
<td style="text-align:left">(24, 2, 1, 611, 4096)</td>
<td style="text-align:right">9.24</td>
<td style="text-align:right">12.65</td>
<td style="text-align:right">23.82</td>
</tr>
<tr>
<td style="text-align:right">123</td>
<td style="text-align:left">(24, 2, 1, 616, 4096)</td>
<td style="text-align:right">9.32</td>
<td style="text-align:right">12.71</td>
<td style="text-align:right">24.02</td>
</tr>
<tr>
<td style="text-align:right">124</td>
<td style="text-align:left">(24, 2, 1, 621, 4096)</td>
<td style="text-align:right">9.4</td>
<td style="text-align:right">12.82</td>
<td style="text-align:right">24.28</td>
</tr>
<tr>
<td style="text-align:right">125</td>
<td style="text-align:left">(24, 2, 1, 626, 4096)</td>
<td style="text-align:right">9.46</td>
<td style="text-align:right">12.94</td>
<td style="text-align:right">133.27</td>
</tr>
<tr>
<td style="text-align:right">126</td>
<td style="text-align:left">(24, 2, 1, 631, 4096)</td>
<td style="text-align:right">6.82</td>
<td style="text-align:right">13.13</td>
<td style="text-align:right">24.65</td>
</tr>
<tr>
<td style="text-align:right">127</td>
<td style="text-align:left">(24, 2, 1, 636, 4096)</td>
<td style="text-align:right">7</td>
<td style="text-align:right">163.93</td>
<td style="text-align:right">24.81</td>
</tr>
<tr>
<td style="text-align:right">128</td>
<td style="text-align:left">(24, 2, 1, 641, 4096)</td>
<td style="text-align:right">6.92</td>
<td style="text-align:right">13.23</td>
<td style="text-align:right">177.68</td>
</tr>
<tr>
<td style="text-align:right">129</td>
<td style="text-align:left">(24, 2, 1, 646, 4096)</td>
<td style="text-align:right">6.97</td>
<td style="text-align:right">13.35</td>
<td style="text-align:right">25.13</td>
</tr>
<tr>
<td style="text-align:right">130</td>
<td style="text-align:left">(24, 2, 1, 651, 4096)</td>
<td style="text-align:right">9.56</td>
<td style="text-align:right">107.45</td>
<td style="text-align:right">25.39</td>
</tr>
<tr>
<td style="text-align:right">131</td>
<td style="text-align:left">(24, 2, 1, 656, 4096)</td>
<td style="text-align:right">9.63</td>
<td style="text-align:right">13.55</td>
<td style="text-align:right">101.99</td>
</tr>
<tr>
<td style="text-align:right">132</td>
<td style="text-align:left">(24, 2, 1, 661, 4096)</td>
<td style="text-align:right">9.68</td>
<td style="text-align:right">13.65</td>
<td style="text-align:right">25.76</td>
</tr>
<tr>
<td style="text-align:right">133</td>
<td style="text-align:left">(24, 2, 1, 666, 4096)</td>
<td style="text-align:right">9.75</td>
<td style="text-align:right">54.84</td>
<td style="text-align:right">25.93</td>
</tr>
<tr>
<td style="text-align:right">134</td>
<td style="text-align:left">(24, 2, 1, 671, 4096)</td>
<td style="text-align:right">9.82</td>
<td style="text-align:right">13.86</td>
<td style="text-align:right">57.36</td>
</tr>
<tr>
<td style="text-align:right">135</td>
<td style="text-align:left">(24, 2, 1, 676, 4096)</td>
<td style="text-align:right">9.95</td>
<td style="text-align:right">13.93</td>
<td style="text-align:right">26.47</td>
</tr>
<tr>
<td style="text-align:right">136</td>
<td style="text-align:left">(24, 2, 1, 681, 4096)</td>
<td style="text-align:right">7.37</td>
<td style="text-align:right">14.09</td>
<td style="text-align:right">26.48</td>
</tr>
<tr>
<td style="text-align:right">137</td>
<td style="text-align:left">(24, 2, 1, 686, 4096)</td>
<td style="text-align:right">140.9</td>
<td style="text-align:right">14.22</td>
<td style="text-align:right">26.63</td>
</tr>
<tr>
<td style="text-align:right">138</td>
<td style="text-align:left">(24, 2, 1, 691, 4096)</td>
<td style="text-align:right">7.45</td>
<td style="text-align:right">14.28</td>
<td style="text-align:right">26.9</td>
</tr>
<tr>
<td style="text-align:right">139</td>
<td style="text-align:left">(24, 2, 1, 696, 4096)</td>
<td style="text-align:right">7.51</td>
<td style="text-align:right">14.37</td>
<td style="text-align:right">27.32</td>
</tr>
<tr>
<td style="text-align:right">140</td>
<td style="text-align:left">(24, 2, 1, 701, 4096)</td>
<td style="text-align:right">7.55</td>
<td style="text-align:right">14.46</td>
<td style="text-align:right">27.26</td>
</tr>
<tr>
<td style="text-align:right">141</td>
<td style="text-align:left">(24, 2, 1, 706, 4096)</td>
<td style="text-align:right">7.7</td>
<td style="text-align:right">124.03</td>
<td style="text-align:right">27.07</td>
</tr>
<tr>
<td style="text-align:right">142</td>
<td style="text-align:left">(24, 2, 1, 711, 4096)</td>
<td style="text-align:right">7.66</td>
<td style="text-align:right">14.68</td>
<td style="text-align:right">27.1</td>
</tr>
<tr>
<td style="text-align:right">143</td>
<td style="text-align:left">(24, 2, 1, 716, 4096)</td>
<td style="text-align:right">7.73</td>
<td style="text-align:right">14.71</td>
<td style="text-align:right">27.47</td>
</tr>
<tr>
<td style="text-align:right">144</td>
<td style="text-align:left">(24, 2, 1, 721, 4096)</td>
<td style="text-align:right">7.78</td>
<td style="text-align:right">14.87</td>
<td style="text-align:right">160.05</td>
</tr>
<tr>
<td style="text-align:right">145</td>
<td style="text-align:left">(24, 2, 1, 726, 4096)</td>
<td style="text-align:right">10.95</td>
<td style="text-align:right">14.91</td>
<td style="text-align:right">27.86</td>
</tr>
<tr>
<td style="text-align:right">146</td>
<td style="text-align:left">(24, 2, 1, 731, 4096)</td>
<td style="text-align:right">11.01</td>
<td style="text-align:right">14.99</td>
<td style="text-align:right">180.19</td>
</tr>
<tr>
<td style="text-align:right">147</td>
<td style="text-align:left">(24, 2, 1, 736, 4096)</td>
<td style="text-align:right">11.05</td>
<td style="text-align:right">15.09</td>
<td style="text-align:right">28.19</td>
</tr>
<tr>
<td style="text-align:right">148</td>
<td style="text-align:left">(24, 2, 1, 741, 4096)</td>
<td style="text-align:right">11.14</td>
<td style="text-align:right">15.2</td>
<td style="text-align:right">129.51</td>
</tr>
<tr>
<td style="text-align:right">149</td>
<td style="text-align:left">(24, 2, 1, 746, 4096)</td>
<td style="text-align:right">11.23</td>
<td style="text-align:right">15.25</td>
<td style="text-align:right">28.61</td>
</tr>
<tr>
<td style="text-align:right">150</td>
<td style="text-align:left">(24, 2, 1, 751, 4096)</td>
<td style="text-align:right">11.32</td>
<td style="text-align:right">15.45</td>
<td style="text-align:right">86.7</td>
</tr>
<tr>
<td style="text-align:right">151</td>
<td style="text-align:left">(24, 2, 1, 756, 4096)</td>
<td style="text-align:right">11.38</td>
<td style="text-align:right">15.6</td>
<td style="text-align:right">29.12</td>
</tr>
<tr>
<td style="text-align:right">152</td>
<td style="text-align:left">(24, 2, 1, 761, 4096)</td>
<td style="text-align:right">11.46</td>
<td style="text-align:right">15.64</td>
<td style="text-align:right">38.48</td>
</tr>
<tr>
<td style="text-align:right">153</td>
<td style="text-align:left">(24, 2, 1, 766, 4096)</td>
<td style="text-align:right">11.53</td>
<td style="text-align:right">15.73</td>
<td style="text-align:right">29.29</td>
</tr>
<tr>
<td style="text-align:right">154</td>
<td style="text-align:left">(24, 2, 1, 771, 4096)</td>
<td style="text-align:right">11.61</td>
<td style="text-align:right">15.77</td>
<td style="text-align:right">29.51</td>
</tr>
<tr>
<td style="text-align:right">155</td>
<td style="text-align:left">(24, 2, 1, 776, 4096)</td>
<td style="text-align:right">11.65</td>
<td style="text-align:right">15.94</td>
<td style="text-align:right">29.56</td>
</tr>
<tr>
<td style="text-align:right">156</td>
<td style="text-align:left">(24, 2, 1, 781, 4096)</td>
<td style="text-align:right">11.72</td>
<td style="text-align:right">15.95</td>
<td style="text-align:right">30.13</td>
</tr>
<tr>
<td style="text-align:right">157</td>
<td style="text-align:left">(24, 2, 1, 786, 4096)</td>
<td style="text-align:right">11.81</td>
<td style="text-align:right">16.21</td>
<td style="text-align:right">29.89</td>
</tr>
<tr>
<td style="text-align:right">158</td>
<td style="text-align:left">(24, 2, 1, 791, 4096)</td>
<td style="text-align:right">11.87</td>
<td style="text-align:right">16.14</td>
<td style="text-align:right">30.07</td>
</tr>
<tr>
<td style="text-align:right">159</td>
<td style="text-align:left">(24, 2, 1, 796, 4096)</td>
<td style="text-align:right">11.94</td>
<td style="text-align:right">16.29</td>
<td style="text-align:right">30.48</td>
</tr>
<tr>
<td style="text-align:right">160</td>
<td style="text-align:left">(24, 2, 1, 801, 4096)</td>
<td style="text-align:right">12.02</td>
<td style="text-align:right">16.37</td>
<td style="text-align:right">30.56</td>
</tr>
<tr>
<td style="text-align:right">161</td>
<td style="text-align:left">(24, 2, 1, 806, 4096)</td>
<td style="text-align:right">12.12</td>
<td style="text-align:right">16.46</td>
<td style="text-align:right">30.89</td>
</tr>
<tr>
<td style="text-align:right">162</td>
<td style="text-align:left">(24, 2, 1, 811, 4096)</td>
<td style="text-align:right">12.17</td>
<td style="text-align:right">112.46</td>
<td style="text-align:right">31.02</td>
</tr>
<tr>
<td style="text-align:right">163</td>
<td style="text-align:left">(24, 2, 1, 816, 4096)</td>
<td style="text-align:right">12.27</td>
<td style="text-align:right">16.63</td>
<td style="text-align:right">31.27</td>
</tr>
<tr>
<td style="text-align:right">164</td>
<td style="text-align:left">(24, 2, 1, 821, 4096)</td>
<td style="text-align:right">12.34</td>
<td style="text-align:right">16.91</td>
<td style="text-align:right">31.31</td>
</tr>
<tr>
<td style="text-align:right">165</td>
<td style="text-align:left">(24, 2, 1, 826, 4096)</td>
<td style="text-align:right">12.4</td>
<td style="text-align:right">16.77</td>
<td style="text-align:right">31.63</td>
</tr>
<tr>
<td style="text-align:right">166</td>
<td style="text-align:left">(24, 2, 1, 831, 4096)</td>
<td style="text-align:right">12.48</td>
<td style="text-align:right">17.02</td>
<td style="text-align:right">32.18</td>
</tr>
<tr>
<td style="text-align:right">167</td>
<td style="text-align:left">(24, 2, 1, 836, 4096)</td>
<td style="text-align:right">12.54</td>
<td style="text-align:right">17.02</td>
<td style="text-align:right">140.98</td>
</tr>
<tr>
<td style="text-align:right">168</td>
<td style="text-align:left">(24, 2, 1, 841, 4096)</td>
<td style="text-align:right">12.6</td>
<td style="text-align:right">17.15</td>
<td style="text-align:right">32.2</td>
</tr>
<tr>
<td style="text-align:right">169</td>
<td style="text-align:left">(24, 2, 1, 846, 4096)</td>
<td style="text-align:right">12.7</td>
<td style="text-align:right">17.23</td>
<td style="text-align:right">32.36</td>
</tr>
<tr>
<td style="text-align:right">170</td>
<td style="text-align:left">(24, 2, 1, 851, 4096)</td>
<td style="text-align:right">12.77</td>
<td style="text-align:right">17.36</td>
<td style="text-align:right">33.89</td>
</tr>
<tr>
<td style="text-align:right">171</td>
<td style="text-align:left">(24, 2, 1, 856, 4096)</td>
<td style="text-align:right">12.84</td>
<td style="text-align:right">17.49</td>
<td style="text-align:right">32.87</td>
</tr>
<tr>
<td style="text-align:right">172</td>
<td style="text-align:left">(24, 2, 1, 861, 4096)</td>
<td style="text-align:right">13</td>
<td style="text-align:right">17.48</td>
<td style="text-align:right">32.95</td>
</tr>
<tr>
<td style="text-align:right">173</td>
<td style="text-align:left">(24, 2, 1, 866, 4096)</td>
<td style="text-align:right">12.98</td>
<td style="text-align:right">17.6</td>
<td style="text-align:right">33.1</td>
</tr>
<tr>
<td style="text-align:right">174</td>
<td style="text-align:left">(24, 2, 1, 871, 4096)</td>
<td style="text-align:right">13.08</td>
<td style="text-align:right">17.69</td>
<td style="text-align:right">140.24</td>
</tr>
<tr>
<td style="text-align:right">175</td>
<td style="text-align:left">(24, 2, 1, 876, 4096)</td>
<td style="text-align:right">13.19</td>
<td style="text-align:right">164.71</td>
<td style="text-align:right">33.59</td>
</tr>
<tr>
<td style="text-align:right">176</td>
<td style="text-align:left">(24, 2, 1, 881, 4096)</td>
<td style="text-align:right">13.19</td>
<td style="text-align:right">17.82</td>
<td style="text-align:right">33.64</td>
</tr>
<tr>
<td style="text-align:right">177</td>
<td style="text-align:left">(24, 2, 1, 886, 4096)</td>
<td style="text-align:right">13.3</td>
<td style="text-align:right">17.91</td>
<td style="text-align:right">34.15</td>
</tr>
<tr>
<td style="text-align:right">178</td>
<td style="text-align:left">(24, 2, 1, 891, 4096)</td>
<td style="text-align:right">13.36</td>
<td style="text-align:right">17.92</td>
<td style="text-align:right">34.11</td>
</tr>
<tr>
<td style="text-align:right">179</td>
<td style="text-align:left">(24, 2, 1, 896, 4096)</td>
<td style="text-align:right">13.45</td>
<td style="text-align:right">18.14</td>
<td style="text-align:right">47.32</td>
</tr>
<tr>
<td style="text-align:right">180</td>
<td style="text-align:left">(24, 2, 1, 901, 4096)</td>
<td style="text-align:right">13.51</td>
<td style="text-align:right">161.89</td>
<td style="text-align:right">187.27</td>
</tr>
<tr>
<td style="text-align:right">181</td>
<td style="text-align:left">(24, 2, 1, 906, 4096)</td>
<td style="text-align:right">13.55</td>
<td style="text-align:right">137.23</td>
<td style="text-align:right">185.95</td>
</tr>
<tr>
<td style="text-align:right">182</td>
<td style="text-align:left">(24, 2, 1, 911, 4096)</td>
<td style="text-align:right">13.64</td>
<td style="text-align:right">132.53</td>
<td style="text-align:right">188.22</td>
</tr>
<tr>
<td style="text-align:right">183</td>
<td style="text-align:left">(24, 2, 1, 916, 4096)</td>
<td style="text-align:right">13.71</td>
<td style="text-align:right">126.04</td>
<td style="text-align:right">186.95</td>
</tr>
<tr>
<td style="text-align:right">184</td>
<td style="text-align:left">(24, 2, 1, 921, 4096)</td>
<td style="text-align:right">13.78</td>
<td style="text-align:right">119.1</td>
<td style="text-align:right">159.75</td>
</tr>
<tr>
<td style="text-align:right">185</td>
<td style="text-align:left">(24, 2, 1, 926, 4096)</td>
<td style="text-align:right">13.87</td>
<td style="text-align:right">103.94</td>
<td style="text-align:right">148.76</td>
</tr>
<tr>
<td style="text-align:right">186</td>
<td style="text-align:left">(24, 2, 1, 931, 4096)</td>
<td style="text-align:right">13.92</td>
<td style="text-align:right">107.04</td>
<td style="text-align:right">144.25</td>
</tr>
<tr>
<td style="text-align:right">187</td>
<td style="text-align:left">(24, 2, 1, 936, 4096)</td>
<td style="text-align:right">14.03</td>
<td style="text-align:right">91.58</td>
<td style="text-align:right">146.02</td>
</tr>
<tr>
<td style="text-align:right">188</td>
<td style="text-align:left">(24, 2, 1, 941, 4096)</td>
<td style="text-align:right">14.13</td>
<td style="text-align:right">91.43</td>
<td style="text-align:right">142.19</td>
</tr>
<tr>
<td style="text-align:right">189</td>
<td style="text-align:left">(24, 2, 1, 946, 4096)</td>
<td style="text-align:right">14.16</td>
<td style="text-align:right">96.95</td>
<td style="text-align:right">139.02</td>
</tr>
<tr>
<td style="text-align:right">190</td>
<td style="text-align:left">(24, 2, 1, 951, 4096)</td>
<td style="text-align:right">14.14</td>
<td style="text-align:right">90.7</td>
<td style="text-align:right">134</td>
</tr>
<tr>
<td style="text-align:right">191</td>
<td style="text-align:left">(24, 2, 1, 956, 4096)</td>
<td style="text-align:right">14.14</td>
<td style="text-align:right">83.12</td>
<td style="text-align:right">128.91</td>
</tr>
<tr>
<td style="text-align:right">192</td>
<td style="text-align:left">(24, 2, 1, 961, 4096)</td>
<td style="text-align:right">14.21</td>
<td style="text-align:right">67.87</td>
<td style="text-align:right">121.61</td>
</tr>
<tr>
<td style="text-align:right">193</td>
<td style="text-align:left">(24, 2, 1, 966, 4096)</td>
<td style="text-align:right">14.29</td>
<td style="text-align:right">74.24</td>
<td style="text-align:right">113.4</td>
</tr>
<tr>
<td style="text-align:right">194</td>
<td style="text-align:left">(24, 2, 1, 971, 4096)</td>
<td style="text-align:right">14.38</td>
<td style="text-align:right">67.69</td>
<td style="text-align:right">112.32</td>
</tr>
<tr>
<td style="text-align:right">195</td>
<td style="text-align:left">(24, 2, 1, 976, 4096)</td>
<td style="text-align:right">14.41</td>
<td style="text-align:right">52.07</td>
<td style="text-align:right">98.98</td>
</tr>
<tr>
<td style="text-align:right">196</td>
<td style="text-align:left">(24, 2, 1, 981, 4096)</td>
<td style="text-align:right">14.48</td>
<td style="text-align:right">43.74</td>
<td style="text-align:right">96.83</td>
</tr>
<tr>
<td style="text-align:right">197</td>
<td style="text-align:left">(24, 2, 1, 986, 4096)</td>
<td style="text-align:right">14.57</td>
<td style="text-align:right">42.39</td>
<td style="text-align:right">95.48</td>
</tr>
<tr>
<td style="text-align:right">198</td>
<td style="text-align:left">(24, 2, 1, 991, 4096)</td>
<td style="text-align:right">14.64</td>
<td style="text-align:right">43.3</td>
<td style="text-align:right">89.24</td>
</tr>
<tr>
<td style="text-align:right">199</td>
<td style="text-align:left">(24, 2, 1, 996, 4096)</td>
<td style="text-align:right">14.76</td>
<td style="text-align:right">38.1</td>
<td style="text-align:right">87.16</td>
</tr>
<tr>
<td style="text-align:right">200</td>
<td style="text-align:left">(24, 2, 1, 1001, 4096)</td>
<td style="text-align:right">14.79</td>
<td style="text-align:right">34.66</td>
<td style="text-align:right">80.94</td>
</tr>
<tr>
<td style="text-align:right">201</td>
<td style="text-align:left">(24, 2, 1, 1006, 4096)</td>
<td style="text-align:right">14.89</td>
<td style="text-align:right">20.43</td>
<td style="text-align:right">75.97</td>
</tr>
<tr>
<td style="text-align:right">202</td>
<td style="text-align:left">(24, 2, 1, 1011, 4096)</td>
<td style="text-align:right">14.68</td>
<td style="text-align:right">23.79</td>
<td style="text-align:right">63.71</td>
</tr>
<tr>
<td style="text-align:right">203</td>
<td style="text-align:left">(24, 2, 1, 1016, 4096)</td>
<td style="text-align:right">14.82</td>
<td style="text-align:right">20.61</td>
<td style="text-align:right">48.38</td>
</tr>
<tr>
<td style="text-align:right">204</td>
<td style="text-align:left">(24, 2, 1, 1021, 4096)</td>
<td style="text-align:right">14.9</td>
<td style="text-align:right">20.74</td>
<td style="text-align:right">39.35</td>
</tr>
<tr>
<td style="text-align:right">205</td>
<td style="text-align:left">(24, 2, 1, 1026, 4096)</td>
<td style="text-align:right">14.92</td>
<td style="text-align:right">20.91</td>
<td style="text-align:right">39.28</td>
</tr>
<tr>
<td style="text-align:right">206</td>
<td style="text-align:left">(24, 2, 1, 1031, 4096)</td>
<td style="text-align:right">15</td>
<td style="text-align:right">20.84</td>
<td style="text-align:right">39.39</td>
</tr>
<tr>
<td style="text-align:right">207</td>
<td style="text-align:left">(24, 2, 1, 1036, 4096)</td>
<td style="text-align:right">14.92</td>
<td style="text-align:right">21.11</td>
<td style="text-align:right">39.48</td>
</tr>
<tr>
<td style="text-align:right">208</td>
<td style="text-align:left">(24, 2, 1, 1041, 4096)</td>
<td style="text-align:right">14.93</td>
<td style="text-align:right">21.2</td>
<td style="text-align:right">39.67</td>
</tr>
<tr>
<td style="text-align:right">209</td>
<td style="text-align:left">(24, 2, 1, 1046, 4096)</td>
<td style="text-align:right">15.01</td>
<td style="text-align:right">21.29</td>
<td style="text-align:right">39.89</td>
</tr>
</tbody>
</table>
</div><p>My questions are</p>
<ol>
<li>Am I setting the experiment correctly? What factors do I miss?</li>
<li>What is the fastest way to transfer a tensor from CPU to GPU on A100?</li>
</ol>