Why are `torch.bool`'s elements 1 byte and not 1 bit?

Geremia: <p>For example, <code>torch.rand(10,10).round().bool().element_size()</code> returns 1 byte (not 0.25 bytes = 1 bit).</p>
<p>Is 1 byte PyTorch’s smallest quantization amount?</p>

Geremia: <p><a href="https://chatgpt.com/share/b0400480-299c-4ef0-8e92-379f2f170278" rel="noopener nofollow ugc">This</a> is a pretty good answer:</p>
<blockquote>
<p>In PyTorch, <code>torch.bool</code> elements occupy 1 byte (8 bits) instead of 1 bit due to the way memory is managed in modern computing architectures. There are several reasons for this design choice:</p>
<ol>
<li>
<p><strong>Alignment and Access Efficiency</strong>:</p>
<ul>
<li>Modern processors are optimized for accessing data aligned to byte boundaries. Accessing single bits would require additional operations to extract the bit from a byte, which would slow down computation.</li>
<li>Byte-aligned data ensures that memory accesses are efficient and can be performed quickly.</li>
</ul>
</li>
<li>
<p><strong>Simplicity in Implementation</strong>:</p>
<ul>
<li>Using 1 byte per boolean simplifies the implementation of tensor operations. Bitwise operations on arrays of bits would require more complex logic to handle individual bit manipulation.</li>
<li>Storing booleans as bytes allows the use of existing memory and data handling mechanisms without needing special cases for bit-level operations.</li>
</ul>
</li>
<li>
<p><strong>Compatibility</strong>:</p>
<ul>
<li>Many libraries and hardware interfaces expect data to be byte-aligned. Using 1 byte per boolean ensures compatibility with other systems and libraries.</li>
<li>This design choice aligns with how other programming languages and libraries often handle boolean arrays, providing consistency across different tools and environments.</li>
</ul>
</li>
</ol>
<h3>Example</h3>
<p>To illustrate the memory usage difference, here is an example with boolean and <code>uint8</code> tensors:</p>
<pre data-code-wrap="python"><code class="lang-python">import torch

# Boolean tensor representing 16 bits
bool_vector = torch.tensor([True, True, False, False, True, False, True, False,
                            False, True, True, True, False, True, True, False], dtype=torch.bool)

print("Boolean tensor:", bool_vector)
print("Size of boolean tensor elements:", bool_vector.element_size())
print("Total memory size of boolean tensor:", bool_vector.element_size() * bool_vector.nelement(), "bytes")

# `uint8` tensor representing the same 16 bits
uint8_vector = torch.tensor([0b11001010, 0b01110110], dtype=torch.uint8)

print("\nuint8 tensor:", uint8_vector)
print("Size of uint8 tensor elements:", uint8_vector.element_size())
print("Total memory size of uint8 tensor:", uint8_vector.element_size() * uint8_vector.nelement(), "bytes")
</code></pre>
<h3>Output</h3>
<pre data-code-wrap="plaintext"><code class="lang-plaintext">Boolean tensor: tensor([ True,  True, False, False,  True, False,  True, False, False,  True,  True,  True, False,  True,  True, False])
Size of boolean tensor elements: 1
Total memory size of boolean tensor: 16 bytes

uint8 tensor: tensor([202, 118], dtype=torch.uint8)
Size of uint8 tensor elements: 1
Total memory size of uint8 tensor: 2 bytes
</code></pre>
<p>In this example, the boolean tensor uses 16 bytes of memory (one byte per boolean), while the <code>uint8</code> tensor uses only 2 bytes to store the same number of bits.</p>
<h3>Conclusion</h3>
<p>The choice to use 1 byte per <code>torch.bool</code> element is a trade-off between memory efficiency and practical performance considerations. By using byte-aligned memory, PyTorch ensures that tensor operations remain fast and compatible with modern hardware and software ecosystems.</p>
</blockquote>

marksaroufim: <p>FWIW this is also true for bools in C++ for that matter, you can bitpack 8 bools into a single byte though <a href="https://www.youtube.com/watch?v=LRJclCVrvQI">https://www.youtube.com/watch?v=LRJclCVrvQI</a></p>