build: parallel build support for modules

Pass the -j and --jobserver-fds (an internal make option) to submakes so
that modules can be built in parallel.

Currently, the modules themselves are built serially, but each individual
module is built in parallel.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Fix spinlock busy-loop to use non-atomic test

Optimize spin_lock() to busy-loop without using an atomic operation in the
contended case. This reduces the need for inter-CPU communication because cache
line does not need to be locked for every iteration of the loop. This improves
spinlock performance for the contended case.

Before:

    Running mutex tests

    Sizes of mutual exclusion primitives:
    Size of lockfree::mutex: 40
    Size of spinlock: 1
    Size of rwlock: 96

    ==== BENCHMARK 1 ====
    Uncontended single-thread lock/unlock cycle:
    Measuring uncontended lockfree::mutex lock/unlock: 29 ns
    Measuring uncontended handoff_stressing_mutex lock/unlock: 51 ns
    Measuring uncontended spinlock lock/unlock: 13 ns
    Measuring uncontended rwlock_read_lock lock/unlock: 63 ns

    ==== BENCHMARK 2 ====
    Contended tests using increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    AAAAAAAAAABBBBBBBBBB
    26 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads
    AAAAAAAAAADCBDCBDCBDCBDCBDCBDCBDCBDCBDCB
    709 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTABDFGJHRKLNOIPMSTCQEADGFJBRKHNOLCQSIMEPTADGBRJKHFNOLCISQMEPTADGRBKHFJNOLISMECPQTADGRBKHFJNOLEMIQSPTCADRGBHJKNOLEIMTSQCPFADBGRJKHNOLEISMPCQTFADBGRJHKNOLIESPCQMTFAGJBRDHKONLIESPTMQCF
    672 ns
    Contended mutex test, spinlock, 2 pinned threads
    ABAAAAAAAAABBBBBBBBB
    52 ns
    Contended mutex test, spinlock, 4 pinned threads
    AAABDCABDBDBDBDBDBDBDBADCBDACACACACACCCC
    252 ns
    Contended mutex test, spinlock, 20 non-pinned threads
    AABBCCDDEEFFGGHHIIJJKKLLMMNNOOPPQQRRSSTTTAABBCCDDEEFFGHIJTGKHLMNOPQRSIJTABCDFEGKLJKRKRHCRKHCFQMRSTKEFCNITNHRAKPCEOBFDTHANRKCEFDTHANJREDFTGAHNDIELAONJDQPOIJQBLPOJIQBMPOJSIQPONBMIQPOMSQLMGSPMSGBLGMLGLSS
    253 ns
    Contended mutex test, rwlock_read_lock, 2 pinned threads
    ABABABABABABABABABAB
    1531 ns
    Contended mutex test, rwlock_read_lock, 4 pinned threads
    ABCDACDBACDBACDBACDBACDBACDBACDBACDBACDB
    2167 ns
    Contended mutex test, rwlock_read_lock, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTBAGCKLONEIDTMJQFSHRPBACGODKLIENMJTQFHSRPBACOGLEDIKTJNMQFSHRPBACITGLEDKJOMNQFSHRPBACILTEOGDJKMNQFHSRPBACIELODGNKJMQFHSRPTBACIELGODKMNJQFHSRPTBAIECLGDKOMJNQFHSPRTBAICLDEKGMONJQFHSPRT
    1432 ns

    ==== MISC TESTS ====

    Trylock tests using spinning_increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    AAABABABABABABAABBBB
    101 ns

    Mutual exclusion test using checker_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads

    2087 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads

    2956 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads

    2259 ns
    mutex tests succeeded

After:

    Running mutex tests

    Sizes of mutual exclusion primitives:
    Size of lockfree::mutex: 40
    Size of spinlock: 1
    Size of rwlock: 96

    ==== BENCHMARK 1 ====
    Uncontended single-thread lock/unlock cycle:
    Measuring uncontended lockfree::mutex lock/unlock: 29 ns
    Measuring uncontended handoff_stressing_mutex lock/unlock: 51 ns
    Measuring uncontended spinlock lock/unlock: 13 ns
    Measuring uncontended rwlock_read_lock lock/unlock: 64 ns

    ==== BENCHMARK 2 ====
    Contended tests using increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    AAAABABABABBABABABBB
    484 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads
    BACDBACDBACDBACDBACDBACDBACDBACDBACDBACD
    902 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTABCEGHILMNROPSQJKFDTABCEGLMIHORFPJKQSTDNABCELGIMOHKPJQSDNTRFABCELIGMHOSPTNKFDQJRABCLEIGMHTKSONDPFJQRABCLEIGMHTKFDSOPJNQRABCLGEIMTHDFKPSJQONRABCGLEIMHTDFKOPQSNJRABCEGLIMHDFTKPSNJOQR
    592 ns
    Contended mutex test, spinlock, 2 pinned threads
    BAAABAAAAABAABBBBBBB
    34 ns
    Contended mutex test, spinlock, 4 pinned threads
    BACDACBADCACBACDACABCDACACBDACBDBDBDBDBD
    68 ns
    Contended mutex test, spinlock, 20 non-pinned threads
    AAABBCCDDEEFFGGHHIIJJKKLLMMNNOOPPQQRRSSTTAAAAAAATDDFEBBGGIIHHJJTKKLLMMOOPPQQRRSCNDFEBGIHJTKLMOPQSCNFERBGDJTKLIMOPQSCNFERBDTKLIMHQOSCNFEPRDLIHJMQOGSCNFEPRDHJMQOSCNFEPTBRIGDLKQOSCNFEPTBRIGLNTBJGMCSHKJHK
    16 ns
    Contended mutex test, rwlock_read_lock, 2 pinned threads
    BABABABABABABABABABA
    1302 ns
    Contended mutex test, rwlock_read_lock, 4 pinned threads
    ACBDABDCABCDABCDACDBACDBACDBADCBADCBACDB
    1773 ns
    Contended mutex test, rwlock_read_lock, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTABDGLNCQIJEFRSOMKHTPABNLDGQICEORSJPHMKFTANBGQDILCOESHPRJFMKTALIQCBDNGSOEHRJPFMKTADIBQCLNGRSHEOPFJKMTAIBDCNGLQRHESOFPJMKTAICBNDLGHRQEOPFSJKMTANBLCIDGHREQPOSFJKMTALBNCDGIRHQEOSJPFMKT
    1249 ns

    ==== MISC TESTS ====

    Trylock tests using spinning_increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    BABABBABABABABBABAAA
    92 ns

    Mutual exclusion test using checker_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads

    2652 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads

    5129 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads

    2004 ns
    mutex tests succeeded

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
timer_task: use external lock

timer_task has an internal mutex lock which protects various timer_task
operations. This patch changes the lock to a reference to an external
lock. This way, instead of doing recursive locking, we can share the lock
with serial_timer_task's mutex lock.

tests/tst-async.c is modified to use the new locking scheme.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mempool: Introduce page_pool

The page pool caches a pool of pages. The number of pages is capped by
_max_size. When we allocate a page, allocate from the pool first if the
pool is not empty. When we free a page, push the page into the pool
instead of really freeing it.

This change reduces page buffer pressure caused by virtio-net
allocations, showign up as contention inside unfill_page_buffer() and
refill_page_buffer(). The virtio-net subsystem allocates pages on one
CPU which are then scattered across all other CPUs and freed to their
per-CPU buffers. To mitigate that, this change enables caching of
pages used by virtio-net.

With this patch, I do not see unfill_page_buffer and refill_page_buffer
showing up in mutex tracing.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Asias He <asias@cloudius-systems.com>