CVE-2016-7798 (Ruby, published 2017-01-30T22:59Z)
Description: value: The openssl gem for Ruby uses the same initialization vector (IV) in GCM Mode (aes-*-gcm) when the IV is set before the key, which makes it easier for context-dependent attackers to bypass the encryption protection mechanism.
Severity: MEDIUM (CVSS v3 base score: 7.5)
Weakness: CWE-326 \u2014 Inadequate Encryption Strength
The product stores or transmits sensitive data using an encryption scheme that is theoretically sound, but is not strong enough for the level of protection required.
Repository: https://github.com/ruby/openssl
Files changed:
ext/openssl/ossl_cipher.c
test/test_cipher.rb
Commit message: cipher: don't set dummy encryption key in Cipher#initialize

Remove the encryption key initialization from Cipher#initialize. This
is effectively a revert of r32723 ("Avoid possible SEGV from AES
encryption/decryption", 2011-07-28).

r32723, which added the key initialization, was a workaround for
Ruby Bug #2768. For some certain ciphers, calling EVP_CipherUpdate()
before setting an encryption key caused segfault. It was not a problem
until OpenSSL implemented GCM mode - the encryption key could be
overridden by repeated calls of EVP_CipherInit_ex(). But, it is not the
case for AES-GCM ciphers. Setting a key, an IV, a key, in this order
causes the IV to be reset to an all-zero IV.

The problem of Bug #2768 persists on the current versions of OpenSSL.
So, make Cipher#update raise an exception if a key is not yet set by the
user. Since encrypting or decrypting without key does not make any
sense, this should not break existing applications.

Users can still call Cipher#key= and Cipher#iv= multiple times with
their own responsibility.

Reference: https://bugs.ruby-lang.org/issues/2768
Reference: https://bugs.ruby-lang.org/issues/8221
Reference: https://github.com/ruby/openssl/issues/49


Vulnerable code:
static ID id_auth_tag_len;
    unsigned char dummy_key[EVP_MAX_KEY_LENGTH] = { 0 };
    /*
     * EVP_CipherInit_ex() allows to specify NULL to key and IV, however some
     * ciphers don't handle well (OpenSSL's bug). [Bug #2768]
     *
     * The EVP which has EVP_CIPH_RAND_KEY flag (such as DES3) allows
     * uninitialized key, but other EVPs (such as AES) does not allow it.
     * Calling EVP_CipherUpdate() without initializing key causes SEGV so we
     * set the data filled with "\0" as the key by default.
     */
    if (EVP_CipherInit_ex(ctx, cipher, NULL, dummy_key, NULL, -1) != 1)
  def test_AES_crush
    500.times do
      assert_nothing_raised("[Bug #2768]") do
        # it caused OpenSSL SEGV by uninitialized key
        OpenSSL::Cipher::AES128.new("ECB").update "." * 17
      end

Fixed code:
static ID id_auth_tag_len, id_key_set;
    if (EVP_CipherInit_ex(ctx, cipher, NULL, NULL, NULL, -1) != 1)
    if (p_key)
	rb_ivar_set(self, id_key_set, Qtrue);

    rb_ivar_set(self, id_key_set, Qtrue);

    if (!RTEST(rb_attr_get(self, id_key_set)))
	ossl_raise(eCipherError, "key not set");

    rb_ivar_set(self, id_key_set, Qtrue);

    id_key_set = rb_intern_const("key_set");
    @c1.random_key
  def test_update_raise_if_key_not_set
    assert_raise(OpenSSL::Cipher::CipherError) do
      # it caused OpenSSL SEGV by uninitialized key [Bug #2768]
      OpenSSL::Cipher::AES128.new("ECB").update "." * 17
  def test_aes_gcm_key_iv_order_issue
    pt = "[ruby/openssl#49]"
    cipher = OpenSSL::Cipher.new("aes-128-gcm").encrypt
    cipher.key = "x" * 16
    cipher.iv = "a" * 12
    ct1 = cipher.update(pt) << cipher.final
    tag1 = cipher.auth_tag

    cipher = OpenSSL::Cipher.new("aes-128-gcm").encrypt
    cipher.iv = "a" * 12
    cipher.key = "x" * 16
    ct2 = cipher.update(pt) << cipher.final
    tag2 = cipher.auth_tag

    assert_equal ct1, ct2
    assert_equal tag1, tag2
  end if has_cipher?("aes-128-gcm")