CVE-2020-24371 (C, published 2020-08-17T17:15Z)
Description: value: lgc.c in Lua 5.4.0 mishandles the interaction between barriers and the sweep phase, leading to a memory access violation involving collectgarbage.
Severity: MEDIUM (CVSS v3 base score: 5.3)
Weakness: CWE-763 \u2014 Release of Invalid Pointer or Reference
The product attempts to return a memory resource to the system, but it calls the wrong release function or calls the appropriate release function incorrectly.
Repository: https://github.com/lua/lua
Files changed:
lgc.c
testes/gengc.lua
Commit message: Fixed bug: barriers cannot be active during sweep

Barriers cannot be active during sweep, even in generational mode.
(Although gen. mode is not incremental, it can hit a barrier when
deleting a thread and closing its upvalues.)  The colors of objects are
being changed during sweep and, therefore, cannot be trusted.


Vulnerable code:
** barrier that moves collector forward, that is, mark the white object
** 'v' being pointed by the black object 'o'. (If in sweep phase, clear
** the black object to white [sweep it] to avoid other barrier calls for
** this same object.) In the generational mode, 'v' must also become
** old, if 'o' is old; however, it cannot be changed directly to OLD,
** because it may still point to non-old objects. So, it is marked as
** OLD0. In the next cycle it will become OLD1, and in the next it
** will finally become OLD (regular old).
    makewhite(g, o);  /* mark main obj. as white to avoid other barriers */
** Mark all values stored in marked open upvalues from non-marked threads.
** (Values from marked threads were already marked when traversing the
** thread.) Remove from the list threads that no longer have upvalues and
** not-marked threads.
          p = next;  /* go to next element */
        else {  /* not touched in this cycle */
          /* else, object is white: just remove it from this list */
  setmetatable(obj, {__gc = gcf, x = "ok"})   -- create its metatable

Fixed code:
** Barrier that moves collector forward, that is, marks the white object
** 'v' being pointed by the black object 'o'.  In the generational
** mode, 'v' must also become old, if 'o' is old; however, it cannot
** be changed directly to OLD, because it may still point to non-old
** objects. So, it is marked as OLD0. In the next cycle it will become
** OLD1, and in the next it will finally become OLD (regular old). By
** then, any object it points to will also be old.  If called in the
** incremental sweep phase, it clears the black object to white (sweep
** it) to avoid other barrier calls for this same object. (That cannot
** be done is generational mode, as its sweep does not distinguish
** whites from deads.)
    if (g->gckind == KGC_INC)  /* incremental mode? */
      makewhite(g, o);  /* mark 'o' as white to avoid other barriers */
** For each non-marked thread, simulates a barrier between each open
** upvalue and its value. (If the thread is collected, the value will be
** assigned to the upvalue, but then it can be too late for the barrier
** to act. The "barrier" does not need to check colors: A non-marked
** thread must be young; upvalues cannot be older than their threads; so
** any visited upvalue must be young too.) Also removes the thread from
** the list, as it was already visited. Removes also threads with no
** upvalues, as they have nothing to be checked. (If the thread gets an
** upvalue later, it will be linked in the list again.)
      lua_assert(!isold(thread) || thread->openupval == NULL);
        lua_assert(getage(uv) <= getage(thread));
** The ages of G_TOUCHED1 and G_TOUCHED2 objects will advance
** in 'correctgraylist'. (That function will also remove objects
** turned white here from any gray list.)
          p = next;  /* keep it in the list and go to next element */
        else {  /* everything else is removed */
          /* white objects are simply removed */
  g->gcstate = GCSswpallgc;
  g->gcstate = GCSswpallgc;
  setmetatable(obj, {__gc = gcf, x = "+"})   -- create its metatable
do   -- another bug in 5.4.0
  local old = {10}
  collectgarbage()   -- make 'old' old
  local co = coroutine.create(
    function ()
      local x = nil
      local f = function ()
                  return x[1]
                end
      x = coroutine.yield(f)
      coroutine.yield()
    end
  )
  local _, f = coroutine.resume(co)   -- create closure over 'x' in coroutine
  collectgarbage("step", 0)   -- make upvalue a survival
  old[1] = {"hello"}    -- 'old' go to grayagain as 'touched1'
  coroutine.resume(co, {123})     -- its value will be new
  co = nil
  collectgarbage("step", 0)   -- hit the barrier
  assert(f() == 123 and old[1][1] == "hello")
  collectgarbage("step", 0)   -- run the collector once more
  -- make sure old[1] was not collected
  assert(f() == 123 and old[1][1] == "hello")
end