CVE-2019-12904 (published 2019-06-20T00:15Z)

In Libgcrypt 1.8.4, the C implementation of AES is vulnerable to a flush-and-reload side-channel attack because physical addresses are available to other processes. (The C implementation is used on platforms where an assembly-language implementation is unavailable.) NOTE: the vendor's position is that the issue report cannot be validated because there is no description of an attack

Severity: MEDIUM
CVSS v2 base score: 4.3 (AV:N/AC:M/Au:N/C:P/I:N/A:N)
CVSS v3 base score: 5.9, MEDIUM (CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N)

Weakness information:
cwe_id: CWE-668
cwe_name: Exposure of Resource to Wrong Sphere
description: The product exposes a resource to the wrong control sphere, providing unintended actors with inappropriate access to the resource.
extended_description: 
            Resources such as files and directories may be inadvertently exposed through mechanisms such as insecure permissions, or when a program accidentally operates on the wrong object. For example, a program may intend that private files can only be provided to a specific user. This effectively defines a control sphere that is intended to prevent attackers from accessing these private files. If the file permissions are insecure, then parties other than the user will be able to access those files.
            A separate control sphere might effectively require that the user can only access the private files, but not any other files on the system. If the program does not ensure that the user is only requesting private files, then the user might be able to access other files on the system.
            In either case, the end result is that a resource has been exposed to the wrong party.
         
         

Fix information:
repo_url: https://github.com/gpg/libgcrypt
commit_details: repo_url: https://github.com/gpg/libgcrypt
author: Jussi Kivilinna
author_date: 2019-05-31 17:18:09+03:00
msg: AES: move look-up tables to .data section and unshare between processes

* cipher/rijndael-internal.h (ATTR_ALIGNED_64): New.
* cipher/rijndael-tables.h (encT): Move to 'enc_tables' structure.
(enc_tables): New structure for encryption table with counters before
and after.
(encT): New macro.
(dec_tables): Add counters before and after encryption table; Move
from .rodata to .data section.
(do_encrypt): Change 'encT' to 'enc_tables.T'.
(do_decrypt): Change '&dec_tables' to 'dec_tables.T'.
* cipher/cipher-gcm.c (prefetch_table): Make inline; Handle input
with length not multiple of 256.
(prefetch_enc, prefetch_dec): Modify pre- and post-table counters
to unshare look-up table pages between processes.
--

GnuPG-bug-id: 4541
Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
repository: repo_url: https://github.com/gpg/libgcrypt
repo_name: gpg/libgcrypt
description: The GNU crypto library. NOTE: Maintainers are not tracking this mirror. Do not make pull requests here, nor comment any commits, submit them usual way to bug tracker (https://www.gnupg.org/documentation/bts.html) or to the mailing list (https://www.gnupg.org/documentation/mailing-lists.html).
repo_language: C
owner: gpg
file_changes: filename: rijndael-internal.h
change_type: ModificationType.MODIFY
diff: @@ -29,11 +29,13 @@
 #define BLOCKSIZE               (128/8)
 
 
-/* Helper macro to force alignment to 16 bytes.  */
+/* Helper macro to force alignment to 16 or 64 bytes.  */
 #ifdef HAVE_GCC_ATTRIBUTE_ALIGNED
 # define ATTR_ALIGNED_16  __attribute__ ((aligned (16)))
+# define ATTR_ALIGNED_64  __attribute__ ((aligned (64)))
 #else
 # define ATTR_ALIGNED_16
+# define ATTR_ALIGNED_64
 #endif
 
 

programming_language: C
filename: rijndael-tables.h
change_type: ModificationType.MODIFY
diff: @@ -21,80 +21,98 @@
 /* To keep the actual implementation at a readable size we use this
    include file to define the tables.  */
 
-static const u32 encT[256] =
+static struct
+{
+  volatile u32 counter_head;
+  u32 cacheline_align[64 / 4 - 1];
+  u32 T[256];
+  volatile u32 counter_tail;
+} enc_tables ATTR_ALIGNED_64 =
   {
-    0xa56363c6, 0x847c7cf8, 0x997777ee, 0x8d7b7bf6,
-    0x0df2f2ff, 0xbd6b6bd6, 0xb16f6fde, 0x54c5c591,
-    0x50303060, 0x03010102, 0xa96767ce, 0x7d2b2b56,
-    0x19fefee7, 0x62d7d7b5, 0xe6abab4d, 0x9a7676ec,
-    0x45caca8f, 0x9d82821f, 0x40c9c989, 0x877d7dfa,
-    0x15fafaef, 0xeb5959b2, 0xc947478e, 0x0bf0f0fb,
-    0xecadad41, 0x67d4d4b3, 0xfda2a25f, 0xeaafaf45,
-    0xbf9c9c23, 0xf7a4a453, 0x967272e4, 0x5bc0c09b,
-    0xc2b7b775, 0x1cfdfde1, 0xae93933d, 0x6a26264c,
-    0x5a36366c, 0x413f3f7e, 0x02f7f7f5, 0x4fcccc83,
-    0x5c343468, 0xf4a5a551, 0x34e5e5d1, 0x08f1f1f9,
-    0x937171e2, 0x73d8d8ab, 0x53313162, 0x3f15152a,
-    0x0c040408, 0x52c7c795, 0x65232346, 0x5ec3c39d,
-    0x28181830, 0xa1969637, 0x0f05050a, 0xb59a9a2f,
-    0x0907070e, 0x36121224, 0x9b80801b, 0x3de2e2df,
-    0x26ebebcd, 0x6927274e, 0xcdb2b27f, 0x9f7575ea,
-    0x1b090912, 0x9e83831d, 0x742c2c58, 0x2e1a1a34,
-    0x2d1b1b36, 0xb26e6edc, 0xee5a5ab4, 0xfba0a05b,
-    0xf65252a4, 0x4d3b3b76, 0x61d6d6b7, 0xceb3b37d,
-    0x7b292952, 0x3ee3e3dd, 0x712f2f5e, 0x97848413,
-    0xf55353a6, 0x68d1d1b9, 0x00000000, 0x2cededc1,
-    0x60202040, 0x1ffcfce3, 0xc8b1b179, 0xed5b5bb6,
-    0xbe6a6ad4, 0x46cbcb8d, 0xd9bebe67, 0x4b393972,
-    0xde4a4a94, 0xd44c4c98, 0xe85858b0, 0x4acfcf85,
-    0x6bd0d0bb, 0x2aefefc5, 0xe5aaaa4f, 0x16fbfbed,
-    0xc5434386, 0xd74d4d9a, 0x55333366, 0x94858511,
-    0xcf45458a, 0x10f9f9e9, 0x06020204, 0x817f7ffe,
-    0xf05050a0, 0x443c3c78, 0xba9f9f25, 0xe3a8a84b,
-    0xf35151a2, 0xfea3a35d, 0xc0404080, 0x8a8f8f05,
-    0xad92923f, 0xbc9d9d21, 0x48383870, 0x04f5f5f1,
-    0xdfbcbc63, 0xc1b6b677, 0x75dadaaf, 0x63212142,
-    0x30101020, 0x1affffe5, 0x0ef3f3fd, 0x6dd2d2bf,
-    0x4ccdcd81, 0x140c0c18, 0x35131326, 0x2fececc3,
-    0xe15f5fbe, 0xa2979735, 0xcc444488, 0x3917172e,
-    0x57c4c493, 0xf2a7a755, 0x827e7efc, 0x473d3d7a,
-    0xac6464c8, 0xe75d5dba, 0x2b191932, 0x957373e6,
-    0xa06060c0, 0x98818119, 0xd14f4f9e, 0x7fdcdca3,
-    0x66222244, 0x7e2a2a54, 0xab90903b, 0x8388880b,
-    0xca46468c, 0x29eeeec7, 0xd3b8b86b, 0x3c141428,
-    0x79dedea7, 0xe25e5ebc, 0x1d0b0b16, 0x76dbdbad,
-    0x3be0e0db, 0x56323264, 0x4e3a3a74, 0x1e0a0a14,
-    0xdb494992, 0x0a06060c, 0x6c242448, 0xe45c5cb8,
-    0x5dc2c29f, 0x6ed3d3bd, 0xefacac43, 0xa66262c4,
-    0xa8919139, 0xa4959531, 0x37e4e4d3, 0x8b7979f2,
-    0x32e7e7d5, 0x43c8c88b, 0x5937376e, 0xb76d6dda,
-    0x8c8d8d01, 0x64d5d5b1, 0xd24e4e9c, 0xe0a9a949,
-    0xb46c6cd8, 0xfa5656ac, 0x07f4f4f3, 0x25eaeacf,
-    0xaf6565ca, 0x8e7a7af4, 0xe9aeae47, 0x18080810,
-    0xd5baba6f, 0x887878f0, 0x6f25254a, 0x722e2e5c,
-    0x241c1c38, 0xf1a6a657, 0xc7b4b473, 0x51c6c697,
-    0x23e8e8cb, 0x7cdddda1, 0x9c7474e8, 0x211f1f3e,
-    0xdd4b4b96, 0xdcbdbd61, 0x868b8b0d, 0x858a8a0f,
-    0x907070e0, 0x423e3e7c, 0xc4b5b571, 0xaa6666cc,
-    0xd8484890, 0x05030306, 0x01f6f6f7, 0x120e0e1c,
-    0xa36161c2, 0x5f35356a, 0xf95757ae, 0xd0b9b969,
-    0x91868617, 0x58c1c199, 0x271d1d3a, 0xb99e9e27,
-    0x38e1e1d9, 0x13f8f8eb, 0xb398982b, 0x33111122,
-    0xbb6969d2, 0x70d9d9a9, 0x898e8e07, 0xa7949433,
-    0xb69b9b2d, 0x221e1e3c, 0x92878715, 0x20e9e9c9,
-    0x49cece87, 0xff5555aa, 0x78282850, 0x7adfdfa5,
-    0x8f8c8c03, 0xf8a1a159, 0x80898909, 0x170d0d1a,
-    0xdabfbf65, 0x31e6e6d7, 0xc6424284, 0xb86868d0,
-    0xc3414182, 0xb0999929, 0x772d2d5a, 0x110f0f1e,
-    0xcbb0b07b, 0xfc5454a8, 0xd6bbbb6d, 0x3a16162c
+    0,
+    { 0, },
+    {
+      0xa56363c6, 0x847c7cf8, 0x997777ee, 0x8d7b7bf6,
+      0x0df2f2ff, 0xbd6b6bd6, 0xb16f6fde, 0x54c5c591,
+      0x50303060, 0x03010102, 0xa96767ce, 0x7d2b2b56,
+      0x19fefee7, 0x62d7d7b5, 0xe6abab4d, 0x9a7676ec,
+      0x45caca8f, 0x9d82821f, 0x40c9c989, 0x877d7dfa,
+      0x15fafaef, 0xeb5959b2, 0xc947478e, 0x0bf0f0fb,
+      0xecadad41, 0x67d4d4b3, 0xfda2a25f, 0xeaafaf45,
+      0xbf9c9c23, 0xf7a4a453, 0x967272e4, 0x5bc0c09b,
+      0xc2b7b775, 0x1cfdfde1, 0xae93933d, 0x6a26264c,
+      0x5a36366c, 0x413f3f7e, 0x02f7f7f5, 0x4fcccc83,
+      0x5c343468, 0xf4a5a551, 0x34e5e5d1, 0x08f1f1f9,
+      0x937171e2, 0x73d8d8ab, 0x53313162, 0x3f15152a,
+      0x0c040408, 0x52c7c795, 0x65232346, 0x5ec3c39d,
+      0x28181830, 0xa1969637, 0x0f05050a, 0xb59a9a2f,
+      0x0907070e, 0x36121224, 0x9b80801b, 0x3de2e2df,
+      0x26ebebcd, 0x6927274e, 0xcdb2b27f, 0x9f7575ea,
+      0x1b090912, 0x9e83831d, 0x742c2c58, 0x2e1a1a34,
+      0x2d1b1b36, 0xb26e6edc, 0xee5a5ab4, 0xfba0a05b,
+      0xf65252a4, 0x4d3b3b76, 0x61d6d6b7, 0xceb3b37d,
+      0x7b292952, 0x3ee3e3dd, 0x712f2f5e, 0x97848413,
+      0xf55353a6, 0x68d1d1b9, 0x00000000, 0x2cededc1,
+      0x60202040, 0x1ffcfce3, 0xc8b1b179, 0xed5b5bb6,
+      0xbe6a6ad4, 0x46cbcb8d, 0xd9bebe67, 0x4b393972,
+      0xde4a4a94, 0xd44c4c98, 0xe85858b0, 0x4acfcf85,
+      0x6bd0d0bb, 0x2aefefc5, 0xe5aaaa4f, 0x16fbfbed,
+      0xc5434386, 0xd74d4d9a, 0x55333366, 0x94858511,
+      0xcf45458a, 0x10f9f9e9, 0x06020204, 0x817f7ffe,
+      0xf05050a0, 0x443c3c78, 0xba9f9f25, 0xe3a8a84b,
+      0xf35151a2, 0xfea3a35d, 0xc0404080, 0x8a8f8f05,
+      0xad92923f, 0xbc9d9d21, 0x48383870, 0x04f5f5f1,
+      0xdfbcbc63, 0xc1b6b677, 0x75dadaaf, 0x63212142,
+      0x30101020, 0x1affffe5, 0x0ef3f3fd, 0x6dd2d2bf,
+      0x4ccdcd81, 0x140c0c18, 0x35131326, 0x2fececc3,
+      0xe15f5fbe, 0xa2979735, 0xcc444488, 0x3917172e,
+      0x57c4c493, 0xf2a7a755, 0x827e7efc, 0x473d3d7a,
+      0xac6464c8, 0xe75d5dba, 0x2b191932, 0x957373e6,
+      0xa06060c0, 0x98818119, 0xd14f4f9e, 0x7fdcdca3,
+      0x66222244, 0x7e2a2a54, 0xab90903b, 0x8388880b,
+      0xca46468c, 0x29eeeec7, 0xd3b8b86b, 0x3c141428,
+      0x79dedea7, 0xe25e5ebc, 0x1d0b0b16, 0x76dbdbad,
+      0x3be0e0db, 0x56323264, 0x4e3a3a74, 0x1e0a0a14,
+      0xdb494992, 0x0a06060c, 0x6c242448, 0xe45c5cb8,
+      0x5dc2c29f, 0x6ed3d3bd, 0xefacac43, 0xa66262c4,
+      0xa8919139, 0xa4959531, 0x37e4e4d3, 0x8b7979f2,
+      0x32e7e7d5, 0x43c8c88b, 0x5937376e, 0xb76d6dda,
+      0x8c8d8d01, 0x64d5d5b1, 0xd24e4e9c, 0xe0a9a949,
+      0xb46c6cd8, 0xfa5656ac, 0x07f4f4f3, 0x25eaeacf,
+      0xaf6565ca, 0x8e7a7af4, 0xe9aeae47, 0x18080810,
+      0xd5baba6f, 0x887878f0, 0x6f25254a, 0x722e2e5c,
+      0x241c1c38, 0xf1a6a657, 0xc7b4b473, 0x51c6c697,
+      0x23e8e8cb, 0x7cdddda1, 0x9c7474e8, 0x211f1f3e,
+      0xdd4b4b96, 0xdcbdbd61, 0x868b8b0d, 0x858a8a0f,
+      0x907070e0, 0x423e3e7c, 0xc4b5b571, 0xaa6666cc,
+      0xd8484890, 0x05030306, 0x01f6f6f7, 0x120e0e1c,
+      0xa36161c2, 0x5f35356a, 0xf95757ae, 0xd0b9b969,
+      0x91868617, 0x58c1c199, 0x271d1d3a, 0xb99e9e27,
+      0x38e1e1d9, 0x13f8f8eb, 0xb398982b, 0x33111122,
+      0xbb6969d2, 0x70d9d9a9, 0x898e8e07, 0xa7949433,
+      0xb69b9b2d, 0x221e1e3c, 0x92878715, 0x20e9e9c9,
+      0x49cece87, 0xff5555aa, 0x78282850, 0x7adfdfa5,
+      0x8f8c8c03, 0xf8a1a159, 0x80898909, 0x170d0d1a,
+      0xdabfbf65, 0x31e6e6d7, 0xc6424284, 0xb86868d0,
+      0xc3414182, 0xb0999929, 0x772d2d5a, 0x110f0f1e,
+      0xcbb0b07b, 0xfc5454a8, 0xd6bbbb6d, 0x3a16162c
+    },
+    0
   };
 
-static const struct
+#define encT enc_tables.T
+
+static struct
 {
+  volatile u32 counter_head;
+  u32 cacheline_align[64 / 4 - 1];
   u32 T[256];
   byte inv_sbox[256];
-} dec_tables =
+  volatile u32 counter_tail;
+} dec_tables ATTR_ALIGNED_64 =
   {
+    0,
+    { 0, },
     {
       0x50a7f451, 0x5365417e, 0xc3a4171a, 0x965e273a,
       0xcb6bab3b, 0xf1459d1f, 0xab58faac, 0x9303e34b,
@@ -194,7 +212,8 @@ static const struct
       0xc8,0xeb,0xbb,0x3c,0x83,0x53,0x99,0x61,
       0x17,0x2b,0x04,0x7e,0xba,0x77,0xd6,0x26,
       0xe1,0x69,0x14,0x63,0x55,0x21,0x0c,0x7d
-    }
+    },
+    0
   };
 
 #define decT dec_tables.T

programming_language: C
filename: rijndael.c
change_type: ModificationType.MODIFY
diff: @@ -218,11 +218,11 @@ static const char *selftest(void);
 
 
 /* Prefetching for encryption/decryption tables. */
-static void prefetch_table(const volatile byte *tab, size_t len)
+static inline void prefetch_table(const volatile byte *tab, size_t len)
 {
   size_t i;
 
-  for (i = 0; i < len; i += 8 * 32)
+  for (i = 0; len - i >= 8 * 32; i += 8 * 32)
     {
       (void)tab[i + 0 * 32];
       (void)tab[i + 1 * 32];
@@ -233,17 +233,37 @@ static void prefetch_table(const volatile byte *tab, size_t len)
       (void)tab[i + 6 * 32];
       (void)tab[i + 7 * 32];
     }
+  for (; i < len; i += 32)
+    {
+      (void)tab[i];
+    }
 
   (void)tab[len - 1];
 }
 
 static void prefetch_enc(void)
 {
-  prefetch_table((const void *)encT, sizeof(encT));
+  /* Modify counters to trigger copy-on-write and unsharing if physical pages
+   * of look-up table are shared between processes.  Modifying counters also
+   * causes checksums for pages to change and hint same-page merging algorithm
+   * that these pages are frequently changing.  */
+  enc_tables.counter_head++;
+  enc_tables.counter_tail++;
+
+  /* Prefetch look-up tables to cache.  */
+  prefetch_table((const void *)&enc_tables, sizeof(enc_tables));
 }
 
 static void prefetch_dec(void)
 {
+  /* Modify counters to trigger copy-on-write and unsharing if physical pages
+   * of look-up table are shared between processes.  Modifying counters also
+   * causes checksums for pages to change and hint same-page merging algorithm
+   * that these pages are frequently changing.  */
+  dec_tables.counter_head++;
+  dec_tables.counter_tail++;
+
+  /* Prefetch look-up tables to cache.  */
   prefetch_table((const void *)&dec_tables, sizeof(dec_tables));
 }
 
@@ -765,9 +785,10 @@ do_encrypt (const RIJNDAEL_context *ctx,
 {
 #ifdef USE_AMD64_ASM
   return _gcry_aes_amd64_encrypt_block(ctx->keyschenc, bx, ax, ctx->rounds,
-				       encT);
+				       enc_tables.T);
 #elif defined(USE_ARM_ASM)
-  return _gcry_aes_arm_encrypt_block(ctx->keyschenc, bx, ax, ctx->rounds, encT);
+  return _gcry_aes_arm_encrypt_block(ctx->keyschenc, bx, ax, ctx->rounds,
+				     enc_tables.T);
 #else
   return do_encrypt_fn (ctx, bx, ax);
 #endif /* !USE_ARM_ASM && !USE_AMD64_ASM*/
@@ -1123,10 +1144,10 @@ do_decrypt (const RIJNDAEL_context *ctx, unsigned char *bx,
 {
 #ifdef USE_AMD64_ASM
   return _gcry_aes_amd64_decrypt_block(ctx->keyschdec, bx, ax, ctx->rounds,
-				       &dec_tables);
+				       dec_tables.T);
 #elif defined(USE_ARM_ASM)
   return _gcry_aes_arm_decrypt_block(ctx->keyschdec, bx, ax, ctx->rounds,
-				     &dec_tables);
+				     dec_tables.T);
 #else
   return do_decrypt_fn (ctx, bx, ax);
 #endif /*!USE_ARM_ASM && !USE_AMD64_ASM*/

programming_language: C
repo_url: https://github.com/gpg/libgcrypt
commit_details: repo_url: https://github.com/gpg/libgcrypt
author: Jussi Kivilinna
author_date: 2019-05-31 17:27:25+03:00
msg: GCM: move look-up table to .data section and unshare between processes

* cipher/cipher-gcm.c (ATTR_ALIGNED_64): New.
(gcmR): Move to 'gcm_table' structure.
(gcm_table): New structure for look-up table with counters before and
after.
(gcmR): New macro.
(prefetch_table): Handle input with length not multiple of 256.
(do_prefetch_tables): Modify pre- and post-table counters to unshare
look-up table pages between processes.
--

GnuPG-bug-id: 4541
Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
repository: repo_url: https://github.com/gpg/libgcrypt
repo_name: gpg/libgcrypt
description: The GNU crypto library. NOTE: Maintainers are not tracking this mirror. Do not make pull requests here, nor comment any commits, submit them usual way to bug tracker (https://www.gnupg.org/documentation/bts.html) or to the mailing list (https://www.gnupg.org/documentation/mailing-lists.html).
repo_language: C
owner: gpg
file_changes: filename: cipher-gcm.c
change_type: ModificationType.MODIFY
diff: @@ -30,6 +30,14 @@
 #include "./cipher-internal.h"
 
 
+/* Helper macro to force alignment to 16 or 64 bytes.  */
+#ifdef HAVE_GCC_ATTRIBUTE_ALIGNED
+# define ATTR_ALIGNED_64  __attribute__ ((aligned (64)))
+#else
+# define ATTR_ALIGNED_64
+#endif
+
+
 #ifdef GCM_USE_INTEL_PCLMUL
 extern void _gcry_ghash_setup_intel_pclmul (gcry_cipher_hd_t c);
 
@@ -83,40 +91,54 @@ ghash_armv7_neon (gcry_cipher_hd_t c, byte *result, const byte *buf,
 
 
 #ifdef GCM_USE_TABLES
-static const u16 gcmR[256] = {
-  0x0000, 0x01c2, 0x0384, 0x0246, 0x0708, 0x06ca, 0x048c, 0x054e,
-  0x0e10, 0x0fd2, 0x0d94, 0x0c56, 0x0918, 0x08da, 0x0a9c, 0x0b5e,
-  0x1c20, 0x1de2, 0x1fa4, 0x1e66, 0x1b28, 0x1aea, 0x18ac, 0x196e,
-  0x1230, 0x13f2, 0x11b4, 0x1076, 0x1538, 0x14fa, 0x16bc, 0x177e,
-  0x3840, 0x3982, 0x3bc4, 0x3a06, 0x3f48, 0x3e8a, 0x3ccc, 0x3d0e,
-  0x3650, 0x3792, 0x35d4, 0x3416, 0x3158, 0x309a, 0x32dc, 0x331e,
-  0x2460, 0x25a2, 0x27e4, 0x2626, 0x2368, 0x22aa, 0x20ec, 0x212e,
-  0x2a70, 0x2bb2, 0x29f4, 0x2836, 0x2d78, 0x2cba, 0x2efc, 0x2f3e,
-  0x7080, 0x7142, 0x7304, 0x72c6, 0x7788, 0x764a, 0x740c, 0x75ce,
-  0x7e90, 0x7f52, 0x7d14, 0x7cd6, 0x7998, 0x785a, 0x7a1c, 0x7bde,
-  0x6ca0, 0x6d62, 0x6f24, 0x6ee6, 0x6ba8, 0x6a6a, 0x682c, 0x69ee,
-  0x62b0, 0x6372, 0x6134, 0x60f6, 0x65b8, 0x647a, 0x663c, 0x67fe,
-  0x48c0, 0x4902, 0x4b44, 0x4a86, 0x4fc8, 0x4e0a, 0x4c4c, 0x4d8e,
-  0x46d0, 0x4712, 0x4554, 0x4496, 0x41d8, 0x401a, 0x425c, 0x439e,
-  0x54e0, 0x5522, 0x5764, 0x56a6, 0x53e8, 0x522a, 0x506c, 0x51ae,
-  0x5af0, 0x5b32, 0x5974, 0x58b6, 0x5df8, 0x5c3a, 0x5e7c, 0x5fbe,
-  0xe100, 0xe0c2, 0xe284, 0xe346, 0xe608, 0xe7ca, 0xe58c, 0xe44e,
-  0xef10, 0xeed2, 0xec94, 0xed56, 0xe818, 0xe9da, 0xeb9c, 0xea5e,
-  0xfd20, 0xfce2, 0xfea4, 0xff66, 0xfa28, 0xfbea, 0xf9ac, 0xf86e,
-  0xf330, 0xf2f2, 0xf0b4, 0xf176, 0xf438, 0xf5fa, 0xf7bc, 0xf67e,
-  0xd940, 0xd882, 0xdac4, 0xdb06, 0xde48, 0xdf8a, 0xddcc, 0xdc0e,
-  0xd750, 0xd692, 0xd4d4, 0xd516, 0xd058, 0xd19a, 0xd3dc, 0xd21e,
-  0xc560, 0xc4a2, 0xc6e4, 0xc726, 0xc268, 0xc3aa, 0xc1ec, 0xc02e,
-  0xcb70, 0xcab2, 0xc8f4, 0xc936, 0xcc78, 0xcdba, 0xcffc, 0xce3e,
-  0x9180, 0x9042, 0x9204, 0x93c6, 0x9688, 0x974a, 0x950c, 0x94ce,
-  0x9f90, 0x9e52, 0x9c14, 0x9dd6, 0x9898, 0x995a, 0x9b1c, 0x9ade,
-  0x8da0, 0x8c62, 0x8e24, 0x8fe6, 0x8aa8, 0x8b6a, 0x892c, 0x88ee,
-  0x83b0, 0x8272, 0x8034, 0x81f6, 0x84b8, 0x857a, 0x873c, 0x86fe,
-  0xa9c0, 0xa802, 0xaa44, 0xab86, 0xaec8, 0xaf0a, 0xad4c, 0xac8e,
-  0xa7d0, 0xa612, 0xa454, 0xa596, 0xa0d8, 0xa11a, 0xa35c, 0xa29e,
-  0xb5e0, 0xb422, 0xb664, 0xb7a6, 0xb2e8, 0xb32a, 0xb16c, 0xb0ae,
-  0xbbf0, 0xba32, 0xb874, 0xb9b6, 0xbcf8, 0xbd3a, 0xbf7c, 0xbebe,
-};
+static struct
+{
+  volatile u32 counter_head;
+  u32 cacheline_align[64 / 4 - 1];
+  u16 R[256];
+  volatile u32 counter_tail;
+} gcm_table ATTR_ALIGNED_64 =
+  {
+    0,
+    { 0, },
+    {
+      0x0000, 0x01c2, 0x0384, 0x0246, 0x0708, 0x06ca, 0x048c, 0x054e,
+      0x0e10, 0x0fd2, 0x0d94, 0x0c56, 0x0918, 0x08da, 0x0a9c, 0x0b5e,
+      0x1c20, 0x1de2, 0x1fa4, 0x1e66, 0x1b28, 0x1aea, 0x18ac, 0x196e,
+      0x1230, 0x13f2, 0x11b4, 0x1076, 0x1538, 0x14fa, 0x16bc, 0x177e,
+      0x3840, 0x3982, 0x3bc4, 0x3a06, 0x3f48, 0x3e8a, 0x3ccc, 0x3d0e,
+      0x3650, 0x3792, 0x35d4, 0x3416, 0x3158, 0x309a, 0x32dc, 0x331e,
+      0x2460, 0x25a2, 0x27e4, 0x2626, 0x2368, 0x22aa, 0x20ec, 0x212e,
+      0x2a70, 0x2bb2, 0x29f4, 0x2836, 0x2d78, 0x2cba, 0x2efc, 0x2f3e,
+      0x7080, 0x7142, 0x7304, 0x72c6, 0x7788, 0x764a, 0x740c, 0x75ce,
+      0x7e90, 0x7f52, 0x7d14, 0x7cd6, 0x7998, 0x785a, 0x7a1c, 0x7bde,
+      0x6ca0, 0x6d62, 0x6f24, 0x6ee6, 0x6ba8, 0x6a6a, 0x682c, 0x69ee,
+      0x62b0, 0x6372, 0x6134, 0x60f6, 0x65b8, 0x647a, 0x663c, 0x67fe,
+      0x48c0, 0x4902, 0x4b44, 0x4a86, 0x4fc8, 0x4e0a, 0x4c4c, 0x4d8e,
+      0x46d0, 0x4712, 0x4554, 0x4496, 0x41d8, 0x401a, 0x425c, 0x439e,
+      0x54e0, 0x5522, 0x5764, 0x56a6, 0x53e8, 0x522a, 0x506c, 0x51ae,
+      0x5af0, 0x5b32, 0x5974, 0x58b6, 0x5df8, 0x5c3a, 0x5e7c, 0x5fbe,
+      0xe100, 0xe0c2, 0xe284, 0xe346, 0xe608, 0xe7ca, 0xe58c, 0xe44e,
+      0xef10, 0xeed2, 0xec94, 0xed56, 0xe818, 0xe9da, 0xeb9c, 0xea5e,
+      0xfd20, 0xfce2, 0xfea4, 0xff66, 0xfa28, 0xfbea, 0xf9ac, 0xf86e,
+      0xf330, 0xf2f2, 0xf0b4, 0xf176, 0xf438, 0xf5fa, 0xf7bc, 0xf67e,
+      0xd940, 0xd882, 0xdac4, 0xdb06, 0xde48, 0xdf8a, 0xddcc, 0xdc0e,
+      0xd750, 0xd692, 0xd4d4, 0xd516, 0xd058, 0xd19a, 0xd3dc, 0xd21e,
+      0xc560, 0xc4a2, 0xc6e4, 0xc726, 0xc268, 0xc3aa, 0xc1ec, 0xc02e,
+      0xcb70, 0xcab2, 0xc8f4, 0xc936, 0xcc78, 0xcdba, 0xcffc, 0xce3e,
+      0x9180, 0x9042, 0x9204, 0x93c6, 0x9688, 0x974a, 0x950c, 0x94ce,
+      0x9f90, 0x9e52, 0x9c14, 0x9dd6, 0x9898, 0x995a, 0x9b1c, 0x9ade,
+      0x8da0, 0x8c62, 0x8e24, 0x8fe6, 0x8aa8, 0x8b6a, 0x892c, 0x88ee,
+      0x83b0, 0x8272, 0x8034, 0x81f6, 0x84b8, 0x857a, 0x873c, 0x86fe,
+      0xa9c0, 0xa802, 0xaa44, 0xab86, 0xaec8, 0xaf0a, 0xad4c, 0xac8e,
+      0xa7d0, 0xa612, 0xa454, 0xa596, 0xa0d8, 0xa11a, 0xa35c, 0xa29e,
+      0xb5e0, 0xb422, 0xb664, 0xb7a6, 0xb2e8, 0xb32a, 0xb16c, 0xb0ae,
+      0xbbf0, 0xba32, 0xb874, 0xb9b6, 0xbcf8, 0xbd3a, 0xbf7c, 0xbebe,
+    },
+    0
+  };
+
+#define gcmR gcm_table.R
 
 static inline
 void prefetch_table(const void *tab, size_t len)
@@ -124,7 +146,7 @@ void prefetch_table(const void *tab, size_t len)
   const volatile byte *vtab = tab;
   size_t i;
 
-  for (i = 0; i < len; i += 8 * 32)
+  for (i = 0; len - i >= 8 * 32; i += 8 * 32)
     {
       (void)vtab[i + 0 * 32];
       (void)vtab[i + 1 * 32];
@@ -135,6 +157,10 @@ void prefetch_table(const void *tab, size_t len)
       (void)vtab[i + 6 * 32];
       (void)vtab[i + 7 * 32];
     }
+  for (; i < len; i += 32)
+    {
+      (void)vtab[i];
+    }
 
   (void)vtab[len - 1];
 }
@@ -142,8 +168,16 @@ void prefetch_table(const void *tab, size_t len)
 static inline void
 do_prefetch_tables (const void *gcmM, size_t gcmM_size)
 {
+  /* Modify counters to trigger copy-on-write and unsharing if physical pages
+   * of look-up table are shared between processes.  Modifying counters also
+   * causes checksums for pages to change and hint same-page merging algorithm
+   * that these pages are frequently changing.  */
+  gcm_table.counter_head++;
+  gcm_table.counter_tail++;
+
+  /* Prefetch look-up tables to cache.  */
   prefetch_table(gcmM, gcmM_size);
-  prefetch_table(gcmR, sizeof(gcmR));
+  prefetch_table(&gcm_table, sizeof(gcm_table));
 }
 
 #ifdef GCM_TABLES_USE_U64

programming_language: C