Fix spinlock busy-loop to use non-atomic test

Optimize spin_lock() to busy-loop without using an atomic operation in the
contended case. This reduces the need for inter-CPU communication because cache
line does not need to be locked for every iteration of the loop. This improves
spinlock performance for the contended case.

Before:

    Running mutex tests

    Sizes of mutual exclusion primitives:
    Size of lockfree::mutex: 40
    Size of spinlock: 1
    Size of rwlock: 96

    ==== BENCHMARK 1 ====
    Uncontended single-thread lock/unlock cycle:
    Measuring uncontended lockfree::mutex lock/unlock: 29 ns
    Measuring uncontended handoff_stressing_mutex lock/unlock: 51 ns
    Measuring uncontended spinlock lock/unlock: 13 ns
    Measuring uncontended rwlock_read_lock lock/unlock: 63 ns

    ==== BENCHMARK 2 ====
    Contended tests using increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    AAAAAAAAAABBBBBBBBBB
    26 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads
    AAAAAAAAAADCBDCBDCBDCBDCBDCBDCBDCBDCBDCB
    709 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTABDFGJHRKLNOIPMSTCQEADGFJBRKHNOLCQSIMEPTADGBRJKHFNOLCISQMEPTADGRBKHFJNOLISMECPQTADGRBKHFJNOLEMIQSPTCADRGBHJKNOLEIMTSQCPFADBGRJKHNOLEISMPCQTFADBGRJHKNOLIESPCQMTFAGJBRDHKONLIESPTMQCF
    672 ns
    Contended mutex test, spinlock, 2 pinned threads
    ABAAAAAAAAABBBBBBBBB
    52 ns
    Contended mutex test, spinlock, 4 pinned threads
    AAABDCABDBDBDBDBDBDBDBADCBDACACACACACCCC
    252 ns
    Contended mutex test, spinlock, 20 non-pinned threads
    AABBCCDDEEFFGGHHIIJJKKLLMMNNOOPPQQRRSSTTTAABBCCDDEEFFGHIJTGKHLMNOPQRSIJTABCDFEGKLJKRKRHCRKHCFQMRSTKEFCNITNHRAKPCEOBFDTHANRKCEFDTHANJREDFTGAHNDIELAONJDQPOIJQBLPOJIQBMPOJSIQPONBMIQPOMSQLMGSPMSGBLGMLGLSS
    253 ns
    Contended mutex test, rwlock_read_lock, 2 pinned threads
    ABABABABABABABABABAB
    1531 ns
    Contended mutex test, rwlock_read_lock, 4 pinned threads
    ABCDACDBACDBACDBACDBACDBACDBACDBACDBACDB
    2167 ns
    Contended mutex test, rwlock_read_lock, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTBAGCKLONEIDTMJQFSHRPBACGODKLIENMJTQFHSRPBACOGLEDIKTJNMQFSHRPBACITGLEDKJOMNQFSHRPBACILTEOGDJKMNQFHSRPBACIELODGNKJMQFHSRPTBACIELGODKMNJQFHSRPTBAIECLGDKOMJNQFHSPRTBAICLDEKGMONJQFHSPRT
    1432 ns

    ==== MISC TESTS ====

    Trylock tests using spinning_increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    AAABABABABABABAABBBB
    101 ns

    Mutual exclusion test using checker_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads

    2087 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads

    2956 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads

    2259 ns
    mutex tests succeeded

After:

    Running mutex tests

    Sizes of mutual exclusion primitives:
    Size of lockfree::mutex: 40
    Size of spinlock: 1
    Size of rwlock: 96

    ==== BENCHMARK 1 ====
    Uncontended single-thread lock/unlock cycle:
    Measuring uncontended lockfree::mutex lock/unlock: 29 ns
    Measuring uncontended handoff_stressing_mutex lock/unlock: 51 ns
    Measuring uncontended spinlock lock/unlock: 13 ns
    Measuring uncontended rwlock_read_lock lock/unlock: 64 ns

    ==== BENCHMARK 2 ====
    Contended tests using increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    AAAABABABABBABABABBB
    484 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads
    BACDBACDBACDBACDBACDBACDBACDBACDBACDBACD
    902 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTABCEGHILMNROPSQJKFDTABCEGLMIHORFPJKQSTDNABCELGIMOHKPJQSDNTRFABCELIGMHOSPTNKFDQJRABCLEIGMHTKSONDPFJQRABCLEIGMHTKFDSOPJNQRABCLGEIMTHDFKPSJQONRABCGLEIMHTDFKOPQSNJRABCEGLIMHDFTKPSNJOQR
    592 ns
    Contended mutex test, spinlock, 2 pinned threads
    BAAABAAAAABAABBBBBBB
    34 ns
    Contended mutex test, spinlock, 4 pinned threads
    BACDACBADCACBACDACABCDACACBDACBDBDBDBDBD
    68 ns
    Contended mutex test, spinlock, 20 non-pinned threads
    AAABBCCDDEEFFGGHHIIJJKKLLMMNNOOPPQQRRSSTTAAAAAAATDDFEBBGGIIHHJJTKKLLMMOOPPQQRRSCNDFEBGIHJTKLMOPQSCNFERBGDJTKLIMOPQSCNFERBDTKLIMHQOSCNFEPRDLIHJMQOGSCNFEPRDHJMQOSCNFEPTBRIGDLKQOSCNFEPTBRIGLNTBJGMCSHKJHK
    16 ns
    Contended mutex test, rwlock_read_lock, 2 pinned threads
    BABABABABABABABABABA
    1302 ns
    Contended mutex test, rwlock_read_lock, 4 pinned threads
    ACBDABDCABCDABCDACDBACDBACDBADCBADCBACDB
    1773 ns
    Contended mutex test, rwlock_read_lock, 20 non-pinned threads
    ABCDEFGHIJKLMNOPQRSTABDGLNCQIJEFRSOMKHTPABNLDGQICEORSJPHMKFTANBGQDILCOESHPRJFMKTALIQCBDNGSOEHRJPFMKTADIBQCLNGRSHEOPFJKMTAIBDCNGLQRHESOFPJMKTAICBNDLGHRQEOPFSJKMTANBLCIDGHREQPOSFJKMTALBNCDGIRHQEOSJPFMKT
    1249 ns

    ==== MISC TESTS ====

    Trylock tests using spinning_increment_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads
    BABABBABABABABBABAAA
    92 ns

    Mutual exclusion test using checker_thread:
    Contended mutex test, lockfree::mutex, 2 pinned threads

    2652 ns
    Contended mutex test, lockfree::mutex, 4 pinned threads

    5129 ns
    Contended mutex test, lockfree::mutex, 20 non-pinned threads

    2004 ns
    mutex tests succeeded

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>