CVE-2018-20406 (C, published 2018-12-23T23:29Z)
Description: value: Modules/_pickle.c in Python before 3.7.1 has an integer overflow via a large LONG_BINPUT value that is mishandled during a "resize to twice the size" attempt. This issue might cause memory exhaustion, but is only relevant if the pickle format is used for serializing tens or hundreds of gigabytes of data. This issue is fixed in: v3.4.10, v3.4.10rc1; v3.5.10, v3.5.10rc1, v3.5.7, v3.5.7rc1, v3.5.8, v3.5.8rc1, v3.5.8rc2, v3.5.9; v3.6.10, v3.6.10rc1, v3.6.11, v3.6.11rc1, v3.6.12, v3.6.7, v3.6.7rc1, v3.6.7rc2, v3.6.8, v3.6.8rc1, v3.6.9, v3.6.9rc1; v3.7.1, v3.7.1rc1, v3.7.1rc2, v3.7.2, v3.7.2rc1, v3.7.3, v3.7.3rc1, v3.7.4, v3.7.4rc1, v3.7.4rc2, v3.7.5, v3.7.5rc1, v3.7.6, v3.7.6rc1, v3.7.7, v3.7.7rc1, v3.7.8, v3.7.8rc1, v3.7.9.
Severity: MEDIUM (CVSS v3 base score: 7.5)
Weakness: CWE-190 \u2014 Integer Overflow or Wraparound
The product performs a calculation that can
         produce an integer overflow or wraparound when the logic
         assumes that the resulting value will always be larger than
         the original value. This occurs when an integer value is
         incremented to a value that is too large to store in the
         associated representation. When this occurs, the value may
         become a very small or negative number.
Repository: https://github.com/python/cpython
Files changed:
Modules/_pickle.c
Commit message: closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261)



Vulnerable code:
    Py_ssize_t mt_mask;
    Py_ssize_t mt_used;
    Py_ssize_t mt_allocated;
    Py_ssize_t memo_size;       /* Capacity of the memo array */
    Py_ssize_t memo_len;        /* Number of objects in the memo */
    Py_ssize_t i;
    for (i = 0; i < self->mt_allocated; i++) {
    size_t mask = (size_t)self->mt_mask;
_PyMemoTable_ResizeTable(PyMemoTable *self, Py_ssize_t min_size)
    Py_ssize_t new_size = MT_MINSIZE;
    Py_ssize_t to_process;
    /* Find the smallest valid table size >= min_size. */
    while (new_size < min_size && new_size > 0)
        new_size <<= 1;
    if (new_size <= 0) {
    if (!(self->mt_used * 3 >= (self->mt_mask + 1) * 2))
    return _PyMemoTable_ResizeTable(self,
        (self->mt_used > 50000 ? 2 : 4) * self->mt_used);
_Unpickler_ResizeMemoList(UnpicklerObject *self, Py_ssize_t new_size)
    Py_ssize_t i;
_Unpickler_MemoGet(UnpicklerObject *self, Py_ssize_t idx)
    if (idx < 0 || idx >= self->memo_size)
_Unpickler_MemoPut(UnpicklerObject *self, Py_ssize_t idx, PyObject *value)
    Py_ssize_t i;
    for (i = 0; i < memo->mt_allocated; ++i) {
    Py_ssize_t i;
    Py_ssize_t new_memo_size = 0;
    Py_ssize_t i;
        for (i = 0; i < new_memo_size; i++) {
        i = new_memo_size;
        while (--i >= 0) {

Fixed code:
    size_t mt_mask;
    size_t mt_used;
    size_t mt_allocated;
    size_t memo_size;       /* Capacity of the memo array */
    size_t memo_len;        /* Number of objects in the memo */
    for (size_t i = 0; i < self->mt_allocated; i++) {
    size_t mask = self->mt_mask;
_PyMemoTable_ResizeTable(PyMemoTable *self, size_t min_size)
    size_t new_size = MT_MINSIZE;
    size_t to_process;
    if (min_size > PY_SSIZE_T_MAX) {

    /* Find the smallest valid table size >= min_size. */
    while (new_size < min_size) {
        new_size <<= 1;
    }
    if (SIZE_MAX / 3 >= self->mt_used && self->mt_used * 3 < self->mt_allocated * 2) {
    }
    // self->mt_used is always < PY_SSIZE_T_MAX, so this can't overflow.
    size_t desired_size = (self->mt_used > 50000 ? 2 : 4) * self->mt_used;
    return _PyMemoTable_ResizeTable(self, desired_size);
_Unpickler_ResizeMemoList(UnpicklerObject *self, size_t new_size)
    size_t i;
_Unpickler_MemoGet(UnpicklerObject *self, size_t idx)
    if (idx >= self->memo_size)
_Unpickler_MemoPut(UnpicklerObject *self, size_t idx, PyObject *value)
    for (size_t i = 0; i < memo->mt_allocated; ++i) {
    size_t i;
    size_t new_memo_size = 0;
        for (size_t i = 0; i < new_memo_size; i++) {
        for (size_t i = new_memo_size - 1; i != SIZE_MAX; i--) {