Repository: randombit/botan
Pull request #3189 (2023-01-17 23:49:00)

Implement write_keystream for CTR and SHAKE
Also fix a bug in the SHAKE cipher

codecov-commenter: # [Codecov](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) Report
Base: **88.08**% // Head: **88.08**% // Increases project coverage by **`+0.00%`** :tada:
> Coverage data is based on head [(`5b5baed`)](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) compared to base [(`8c69a25`)](https://codecov.io/gh/randombit/botan/commit/8c69a2513af0868356833d7d9f8806eabb126082?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master    #3189   +/-   ##
=======================================
  Coverage   88.08%   88.08%           
=======================================
  Files         604      604           
  Lines       67326    67397   +71     
  Branches     6733     6747   +14     
=======================================
+ Hits        59301    59365   +64     
- Misses       5203     5221   +18     
+ Partials     2822     2811   -11     
```


| [Impacted Files](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) | Coverage Δ | |
|---|---|---|
| [src/cli/speed.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2NsaS9zcGVlZC5jcHA=) | `90.82% <ø> (ø)` | |
| [src/lib/stream/ctr/ctr.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2xpYi9zdHJlYW0vY3RyL2N0ci5jcHA=) | `92.00% <100.00%> (+1.44%)` | :arrow_up: |
| [src/lib/stream/shake\_cipher/shake\_cipher.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2xpYi9zdHJlYW0vc2hha2VfY2lwaGVyL3NoYWtlX2NpcGhlci5jcHA=) | `94.64% <100.00%> (+3.21%)` | :arrow_up: |
| [src/tests/test\_stream.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3Rlc3Rfc3RyZWFtLmNwcA==) | `89.21% <100.00%> (+3.50%)` | :arrow_up: |
| [src/lib/entropy/rdseed/rdseed.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2xpYi9lbnRyb3B5L3Jkc2VlZC9yZHNlZWQuY3Bw) | `18.18% <0.00%> (-63.64%)` | :arrow_down: |
| [src/fuzzer/barrett.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2Z1enplci9iYXJyZXR0LmNwcA==) | `66.66% <0.00%> (-5.56%)` | :arrow_down: |
| [src/cli/cli\_rng.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2NsaS9jbGlfcm5nLmNwcA==) | `60.00% <0.00%> (-4.00%)` | :arrow_down: |
| [src/lib/utils/cpuid/cpuid\_x86.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2xpYi91dGlscy9jcHVpZC9jcHVpZF94ODYuY3Bw) | `51.35% <0.00%> (-3.32%)` | :arrow_down: |
| [src/lib/utils/cpuid/cpuid.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL2xpYi91dGlscy9jcHVpZC9jcHVpZC5jcHA=) | `65.21% <0.00%> (-0.98%)` | :arrow_down: |
| [src/tests/test\_bigint.cpp](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd#diff-c3JjL3Rlc3RzL3Rlc3RfYmlnaW50LmNwcA==) | `90.86% <0.00%> (+0.04%)` | :arrow_up: |
| ... and [6 more](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/randombit/botan/pull/3189?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jack+Lloyd).


Patch:
diff --git a/src/cli/speed.cpp b/src/cli/speed.cpp
index 4d58db80c..fb4c740cb 100644
--- a/src/cli/speed.cpp
+++ b/src/cli/speed.cpp
@@ -966,6 +966,17 @@ class Speed final : public Command
                }
 
             record_result(encrypt_timer);
+
+            if(verbose())
+               {
+               auto ks_timer = make_timer(cipher.name(), buffer.size(), "write_keystream", provider, buf_size);
+
+               while(ks_timer->under(runtime))
+                  {
+                  ks_timer->run([&]() { cipher.write_keystream(buffer.data(), buffer.size()); });
+                  }
+               record_result(ks_timer);
+               }
             }
          }
 #endif
diff --git a/src/lib/stream/ctr/ctr.cpp b/src/lib/stream/ctr/ctr.cpp
index 7ebc1ec2c..bd79b12b5 100644
--- a/src/lib/stream/ctr/ctr.cpp
+++ b/src/lib/stream/ctr/ctr.cpp
@@ -122,6 +122,38 @@ void CTR_BE::cipher(const uint8_t in[], uint8_t out[], size_t length)
    m_pad_pos += length;
    }
 
+void CTR_BE::write_keystream(uint8_t out[], size_t length)
+   {
+   verify_key_set(m_iv.empty() == false);
+
+   const size_t avail = m_pad.size() - m_pad_pos;
+   const size_t take = std::min(length, avail);
+   copy_mem(out, &m_pad[m_pad_pos], take);
+   length -= take;
+   out += take;
+   m_pad_pos += take;
+
+   while(length >= m_pad.size())
+      {
+      add_counter(m_ctr_blocks);
+      m_cipher->encrypt_n(m_counter.data(), out, m_ctr_blocks);
+
+      length -= m_pad.size();
+      out += m_pad.size();
+      }
+
+   if(m_pad_pos == m_pad.size())
+      {
+      add_counter(m_ctr_blocks);
+      m_cipher->encrypt_n(m_counter.data(), m_pad.data(), m_ctr_blocks);
+      m_pad_pos = 0;
+      }
+
+   copy_mem(out, &m_pad[0], length);
+   m_pad_pos += length;
+   BOTAN_ASSERT_NOMSG(m_pad_pos < m_pad.size());
+   }
+
 void CTR_BE::set_iv(const uint8_t iv[], size_t iv_len)
    {
    if(!valid_iv_length(iv_len))
diff --git a/src/lib/stream/ctr/ctr.h b/src/lib/stream/ctr/ctr.h
index 1721dfb8c..9024023d2 100644
--- a/src/lib/stream/ctr/ctr.h
+++ b/src/lib/stream/ctr/ctr.h
@@ -21,6 +21,8 @@ class CTR_BE final : public StreamCipher
    public:
       void cipher(const uint8_t in[], uint8_t out[], size_t length) override;
 
+      void write_keystream(uint8_t out[], size_t length) override;
+
       void set_iv(const uint8_t iv[], size_t iv_len) override;
 
       size_t default_iv_length() const override;
diff --git a/src/lib/stream/shake_cipher/shake_cipher.cpp b/src/lib/stream/shake_cipher/shake_cipher.cpp
index b715525e0..35ea77833 100644
--- a/src/lib/stream/shake_cipher/shake_cipher.cpp
+++ b/src/lib/stream/shake_cipher/shake_cipher.cpp
@@ -62,6 +62,44 @@ void SHAKE_Cipher::cipher(const uint8_t in[], uint8_t out[], size_t length)
    m_buf_pos += length;
    }
 
+void SHAKE_Cipher::write_keystream(uint8_t out[], size_t length)
+   {
+   verify_key_set(m_state.empty() == false);
+
+   if(m_buf_pos > 0)
+      {
+      const size_t take = std::min(length, m_shake_rate - m_buf_pos);
+      copy_mem(out, &m_buffer[m_buf_pos], take);
+      out += take;
+      length -= take;
+      m_buf_pos += take;
+
+      if(m_buf_pos == m_shake_rate)
+         {
+         SHA_3::permute(m_state.data());
+         m_buf_pos = 0;
+         }
+      }
+
+   if(length == 0)
+      return;
+
+   BOTAN_ASSERT_NOMSG(m_buf_pos == 0);
+
+   while(length >= m_shake_rate)
+      {
+      copy_out_le(out, m_shake_rate, m_state.data());
+      SHA_3::permute(m_state.data());
+      length -= m_shake_rate;
+      out += m_shake_rate;
+      }
+
+   copy_out_le(m_buffer.data(), m_shake_rate, m_state.data());
+
+   copy_mem(out, &m_buffer[0], length);
+   m_buf_pos += length;
+   }
+
 void SHAKE_Cipher::key_schedule(const uint8_t key[], size_t length)
    {
    const size_t SHAKE_BITRATE = m_shake_rate*8;
@@ -72,6 +110,7 @@ void SHAKE_Cipher::key_schedule(const uint8_t key[], size_t length)
    const size_t S_pos = SHA_3::absorb(SHAKE_BITRATE, m_state, 0, key, length);
    SHA_3::finish(SHAKE_BITRATE, m_state, S_pos, 0x1F, 0x80);
    copy_out_le(m_buffer.data(), m_buffer.size(), m_state.data());
+   m_buf_pos = 0;
    }
 
 Key_Length_Specification SHAKE_Cipher::key_spec() const
diff --git a/src/lib/stream/shake_cipher/shake_cipher.h b/src/lib/stream/shake_cipher/shake_cipher.h
index 43d7f38e1..c14a827b2 100644
--- a/src/lib/stream/shake_cipher/shake_cipher.h
+++ b/src/lib/stream/shake_cipher/shake_cipher.h
@@ -28,6 +28,8 @@ class SHAKE_Cipher : public StreamCipher
       */
       void cipher(const uint8_t in[], uint8_t out[], size_t length) override final;
 
+      void write_keystream(uint8_t out[], size_t length) override;
+
       /**
       * Seeking is not supported, this function will throw
       */
diff --git a/src/lib/stream/stream_cipher.h b/src/lib/stream/stream_cipher.h
index f13397818..0b1ad62a2 100644
--- a/src/lib/stream/stream_cipher.h
+++ b/src/lib/stream/stream_cipher.h
@@ -60,6 +60,9 @@ class BOTAN_PUBLIC_API(2,0) StreamCipher : public SymmetricAlgorithm
 
       /**
       * Write keystream bytes to a buffer
+      *
+      * The contents of @p out are ignored/overwritten
+      *
       * @param out the byte array to hold the keystream
       * @param len the length of out in bytes
       */


Test patch:
diff --git a/src/tests/test_stream.cpp b/src/tests/test_stream.cpp
index bea97f9ce..f5a819b64 100644
--- a/src/tests/test_stream.cpp
+++ b/src/tests/test_stream.cpp
@@ -158,16 +158,54 @@ class Stream_Cipher_Tests final : public Text_Based_Test
             result.test_eq(provider, "encrypt", buf, expected);
             }
 
+            {
+            if(nonce.size() == 0)
+               cipher->set_key(key);
+            else
+               cipher->set_iv(nonce.data(), nonce.size());
+            if(seek != 0)
+               cipher->seek(seek);
+            std::vector<uint8_t> buf = input;
+            cipher->encrypt(buf);
+            result.test_eq(provider, "encrypt 2", buf, expected);
+            }
+
             if(!nonce.empty())
                {
-               std::vector<uint8_t> buf = input;
                cipher->set_iv(nonce.data(), nonce.size());
                if(seek != 0)
                   cipher->seek(seek);
+               std::vector<uint8_t> buf = input;
                cipher->encrypt(buf);
                result.test_eq(provider, "second encrypt", buf, expected);
                }
 
+            {
+            cipher->set_key(key);
+
+            cipher->set_iv(nonce.data(), nonce.size());
+
+            if(seek != 0)
+               cipher->seek(seek);
+
+            std::vector<uint8_t> buf(input.size(), 0xAB);
+
+            uint8_t* buf_ptr = buf.data();
+            size_t buf_len = buf.size();
+
+            while(buf_len > 0)
+               {
+               size_t next = std::min<size_t>(buf_len, rng().next_byte());
+               cipher->write_keystream(buf_ptr, next);
+               buf_ptr += next;
+               buf_len -= next;
+               }
+
+            for(size_t i = 0; i != input.size(); ++i)
+               buf[i] ^= input[i];
+            result.test_eq(provider, "write_keystream", buf, expected);
+            }
+
             cipher->clear();
 
             try