Repository: biojppm/rapidyaml (language: python)

Issue:
Some test failures in 0.7.0 when compiled gcc -O2
I was working on updating the [Fedora package](https://src.fedoraproject.org/rpms/rapidyaml) to 0.7.0 and encountered a couple of test failures:

```
97% tests passed, 2 tests failed out of 65
Total Test time (real) =   0.76 sec
The following tests FAILED:
	 41 - ryml-test-scalar_literal (Failed)
	 42 - ryml-test-scalar_folded (Subprocess aborted)
```

I was able to reproduce these in a git checkout on Fedora 40 (`x86_64`) by setting the same build flags and options as the RPM build; here I encountered a third test with failures,

```
99% tests passed, 3 tests failed out of 1644

Total Test time (real) =  21.90 sec

The following tests FAILED:
         41 - ryml-test-scalar_literal (Failed)
         42 - ryml-test-scalar_folded (Subprocess aborted)
        1543 - ryml-test-suite-W4TN-in_yaml-events (Failed)
```

----

To reproduce in a checkout, I imitated the RPM build. This uses the default options from the `%cmake` RPM macro, plus:

- `-GNinja` – because it is faster and has no real disadvantages
- `-DRYML_CXX_STANDARD=14` – for compatibility with the current packaged gtest, which no longer supports C++11; changing this to C++17 makes no difference
- `-DRYML_BUILD_TESTS:BOOL=ON`
- `-DRYML_TEST_FUZZ:BOOL=OFF` – so that we do not have to include the contents of https://github.com/biojppm/rapidyaml-data in the source RPM

That looks like:

```bash
gh repo clone biojppm/rapidyaml
cd rapidyaml
git checkout v0.7.0
git submodule update --init --recursive

CFLAGS="${CFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer }" ; export CFLAGS ;
CXXFLAGS="${CXXFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer }" ; export CXXFLAGS ;
FFLAGS="${FFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/lib64/gfortran/modules }" ; export FFLAGS ; 
FCFLAGS="${FCFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/lib64/gfortran/modules }" ; export FCFLAGS ; 
VALAFLAGS="${VALAFLAGS:--g}" ; export VALAFLAGS ;
RUSTFLAGS="${RUSTFLAGS:--Copt-level=3 -Cdebuginfo=2 -Ccodegen-units=1 -Cstrip=none -Cforce-frame-pointers=yes --cap-lints=warn}" ; export RUSTFLAGS ; 
LDFLAGS="${LDFLAGS:--Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1  }" ; export LDFLAGS ;
LT_SYS_LIBRARY_PATH="${LT_SYS_LIBRARY_PATH:-/usr/lib64:}" ; export LT_SYS_LIBRARY_PATH ;
CC="${CC:-gcc}" ; export CC ;
CXX="${CXX:-g++}" ; export CXX

/usr/bin/cmake \
        -S "." \
        -B "redhat-linux-build" \
        -DCMAKE_C_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DCMAKE_Fortran_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \
        -DCMAKE_INSTALL_DO_STRIP:BOOL=OFF \
        -DCMAKE_INSTALL_PREFIX:PATH=/usr \
        -DINCLUDE_INSTALL_DIR:PATH=/usr/include \
        -DLIB_INSTALL_DIR:PATH=/usr/lib64 \
        -DSYSCONF_INSTALL_DIR:PATH=/etc \
        -DSHARE_INSTALL_PREFIX:PATH=/usr/share \
        -DLIB_SUFFIX=64 \
        -DBUILD_SHARED_LIBS:BOOL=ON \
        -GNinja \
        -DRYML_CXX_STANDARD=14 \
        -DRYML_BUILD_TESTS:BOOL=ON \
        -DRYML_TEST_FUZZ:BOOL=OFF

/usr/bin/cmake --build "redhat-linux-build" -j16 --verbose
/usr/bin/cmake --build "redhat-linux-build" -j16 --verbose --target ryml-test-run-verbose
```

By trial and error, I was able to minimize that to the following:

```bash
gh repo clone biojppm/rapidyaml
cd rapidyaml
git checkout v0.7.0
git submodule update --init --recursive

CXXFLAGS='-O2' /usr/bin/cmake -S . -B build \
        -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DRYML_BUILD_TESTS:BOOL=ON

/usr/bin/cmake --build build -j16 --verbose
/usr/bin/cmake --build build -j16 --verbose --target ryml-test-run-verbose
```

In short, in order to reproduce this, it seems to suffice to set `-O2 -NDEBUG`.
Some test failures in 0.7.0 when compiled gcc -O2
I was working on updating the [Fedora package](https://src.fedoraproject.org/rpms/rapidyaml) to 0.7.0 and encountered a couple of test failures:

```
97% tests passed, 2 tests failed out of 65
Total Test time (real) =   0.76 sec
The following tests FAILED:
	 41 - ryml-test-scalar_literal (Failed)
	 42 - ryml-test-scalar_folded (Subprocess aborted)
```

I was able to reproduce these in a git checkout on Fedora 40 (`x86_64`) by setting the same build flags and options as the RPM build; here I encountered a third test with failures,

```
99% tests passed, 3 tests failed out of 1644

Total Test time (real) =  21.90 sec

The following tests FAILED:
         41 - ryml-test-scalar_literal (Failed)
         42 - ryml-test-scalar_folded (Subprocess aborted)
        1543 - ryml-test-suite-W4TN-in_yaml-events (Failed)
```

----

To reproduce in a checkout, I imitated the RPM build. This uses the default options from the `%cmake` RPM macro, plus:

- `-GNinja` – because it is faster and has no real disadvantages
- `-DRYML_CXX_STANDARD=14` – for compatibility with the current packaged gtest, which no longer supports C++11; changing this to C++17 makes no difference
- `-DRYML_BUILD_TESTS:BOOL=ON`
- `-DRYML_TEST_FUZZ:BOOL=OFF` – so that we do not have to include the contents of https://github.com/biojppm/rapidyaml-data in the source RPM

That looks like:

```bash
gh repo clone biojppm/rapidyaml
cd rapidyaml
git checkout v0.7.0
git submodule update --init --recursive

CFLAGS="${CFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer }" ; export CFLAGS ;
CXXFLAGS="${CXXFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer }" ; export CXXFLAGS ;
FFLAGS="${FFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/lib64/gfortran/modules }" ; export FFLAGS ; 
FCFLAGS="${FCFLAGS:--O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/lib64/gfortran/modules }" ; export FCFLAGS ; 
VALAFLAGS="${VALAFLAGS:--g}" ; export VALAFLAGS ;
RUSTFLAGS="${RUSTFLAGS:--Copt-level=3 -Cdebuginfo=2 -Ccodegen-units=1 -Cstrip=none -Cforce-frame-pointers=yes --cap-lints=warn}" ; export RUSTFLAGS ; 
LDFLAGS="${LDFLAGS:--Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1  }" ; export LDFLAGS ;
LT_SYS_LIBRARY_PATH="${LT_SYS_LIBRARY_PATH:-/usr/lib64:}" ; export LT_SYS_LIBRARY_PATH ;
CC="${CC:-gcc}" ; export CC ;
CXX="${CXX:-g++}" ; export CXX

/usr/bin/cmake \
        -S "." \
        -B "redhat-linux-build" \
        -DCMAKE_C_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DCMAKE_Fortran_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \
        -DCMAKE_INSTALL_DO_STRIP:BOOL=OFF \
        -DCMAKE_INSTALL_PREFIX:PATH=/usr \
        -DINCLUDE_INSTALL_DIR:PATH=/usr/include \
        -DLIB_INSTALL_DIR:PATH=/usr/lib64 \
        -DSYSCONF_INSTALL_DIR:PATH=/etc \
        -DSHARE_INSTALL_PREFIX:PATH=/usr/share \
        -DLIB_SUFFIX=64 \
        -DBUILD_SHARED_LIBS:BOOL=ON \
        -GNinja \
        -DRYML_CXX_STANDARD=14 \
        -DRYML_BUILD_TESTS:BOOL=ON \
        -DRYML_TEST_FUZZ:BOOL=OFF

/usr/bin/cmake --build "redhat-linux-build" -j16 --verbose
/usr/bin/cmake --build "redhat-linux-build" -j16 --verbose --target ryml-test-run-verbose
```

By trial and error, I was able to minimize that to the following:

```bash
gh repo clone biojppm/rapidyaml
cd rapidyaml
git checkout v0.7.0
git submodule update --init --recursive

CXXFLAGS='-O2' /usr/bin/cmake -S . -B build \
        -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" \
        -DRYML_BUILD_TESTS:BOOL=ON

/usr/bin/cmake --build build -j16 --verbose
/usr/bin/cmake --build build -j16 --verbose --target ryml-test-run-verbose
```

In short, in order to reproduce this, it seems to suffice to set `-O2 -NDEBUG`.


Discussion / hints:
I would conjecture that perhaps something somewhere is relying on undefined behavior, and as a result the optimizer ends up generating incorrect code – but an assertion happens to impede the optimization that reveals the problem. This would explain why the tests seem to pass when `NDEBUG` is not defined.
Thanks for the detailed repro, which is helpful.

I was able to reproduce the problem with gcc. But not with clang; didn't try with visual studio. 

FYI during the parser refactor I've [had similar trouble with the visual studio 32 bit optimizer](https://github.com/biojppm/rapidyaml/blob/31d8205d18d749f0410c42b1aff1c3ab3c14494c/src/c4/yml/parse_engine.def.hpp#L3194); none of the sanitizers or static analyzers were reporting any problem there or anywhere else. Like I noted there, I was only able to work around the problem, which I did by moving the problematic code to a separate function.

The outward signs are similar to that problem; if I enable the logs in the parser the failure goes away.

Also, using `-O3` makes the failure go away as well.

Fun times ahead.

> Thanks for the detailed repro, which is helpful.

You’re welcome. It took a few days of intermittently poking at this for me to get from “it breaks in my environment, but I can’t tell you why” to something useful enough to report.

> I was able to reproduce the problem with gcc (but not with clang; didn't try with visual studio).

I’m glad to hear it! At least we have consistency.

> Fun times ahead.

Yes, I always dread reporting bugs like this one because I know how hard they can be to investigate.

I’ll keep Fedora Rawhide at 0.6.0 for now. Please let me know if there’s anything I can do to help (other than actually finding the root cause!)
Well, the dread for these bugs is real, and pervasive. It is a laborious process to trace them to their source.

This command will reproduce all failures:
```
cd ~/proj/rapidyaml ;  \
( export BDIR=440_o2 ; \
  CXXFLAGS='-O2' cmake -S . -B $BDIR -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" -DRYML_BUILD_TESTS:BOOL=ON -DRYML_DBG:BOOL=OFF ; \
  pairs=( \
    "ryml-test-scalar_literal,--gtest_filter=*/YmlT*ix/7" \
    "ryml-test-scalar_folded,--gtest_filter=*/YmlT*ix/0" \
    "ryml-test-suite,W4TN-in_yaml /home/jpmag/proj/extern/yaml-test-suite/W4TN in.yaml --gtest_filter=_/unix_in_arena.1_compare_emitted_events_to_ref_events/0" \
  ) ; \
  for pair in "${pairs[@]}" ; do \
    IFS=, read tgt filt <<< "$pair"  ; \
    echo "$tgt -- $filt" ; \
    cmake --build $BDIR -j --verbose --target $tgt && \
    ./$BDIR/test/$tgt $filt ; \
  done \
) 
```

I'll keep recording relevant info here.
Well, the first two failures seem to be similar, and related to the parsing code: the parser seems to be slurping multiple docs into a single scalar. The error I linked to above is related to scalar filtering (ie happens when the scalar has already been slurped and is being filtered), and at first glance seems to be different from these two.

Here are the failed assertions for the first two failures:

## scalar_literal

```
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:162: Failure
Expected equality of these values:
  actualch.val()
    Which is: 'hello0
there
---
|
 hello1
 there
---
|
  hello2
  there
---
|
ciao3
qua
---
|
    ciao4
    qua
---
|
      ciao5
      qua
---
- |
 hello6
 there
- |
 ciao7
 qua
---
foo: |
 hello8
 there
bar: |
 ciao9
 qua
'
  expectedch.val
    Which is: 'hello0
there
'
Google Test trace:
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:158: 1
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:90: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:89: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:212: comparing: iteration based on the ref children
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:179: 0
/home/usr/proj/rapidyaml/test/test_lib/test_group.cpp:71: comparing parsed tree to ref tree
/home/usr/proj/rapidyaml/test/./test_lib/test_group.def.hpp:13: unix style
/home/usr/proj/rapidyaml/test/test_scalar_literal.cpp:1396: case
```

## scalar_folded

```
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:162: Failure
Expected equality of these values:
  actualch.val()
    Which is: 'hello there --- >
 hello
 there
--- >
  hello
  there
--- > ciao qua --- >
    ciao
    qua
--- >
      ciao
      qua
--- - >
 hello
 there
- >
 ciao
 qua
--- foo: >
 hello
 there
bar: >
 ciao
 qua
'
  expectedch.val
    Which is: 'hello there
'
Google Test trace:
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:158: 1
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:90: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:89: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:212: comparing: iteration based on the ref children
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:179: 0
/home/usr/proj/rapidyaml/test/test_lib/test_group.cpp:71: comparing parsed tree to ref tree
/home/usr/proj/rapidyaml/test/./test_lib/test_group.def.hpp:13: unix style
/home/usr/proj/rapidyaml/test/test_scalar_folded.cpp:1858: case
```

I’m able to get some output from Clang’s static analyzers like this:

```
$ CXXFLAGS='-O2 -g' CXX=clang++ scan-build cmake -S . -B build -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" -DRYML_BUILD_TESTS:BOOL=ON
$ scan-build cmake --build build -j16 --verbose
```

I am attaching the output directory as [scan-build-2024-06-21-132823-3551348-1.zip](https://github.com/user-attachments/files/15931091/scan-build-2024-06-21-132823-3551348-1.zip). It’s based on the latest `master`, 31d8205d18d749f0410c42b1aff1c3ab3c14494c. Hopefully there is something helpful in there.
Just for completeness, I tried the clang-analyzer/scan-build build with `-Og -g` and got exactly the same results. Then I tried omitting `-DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG"` and still got the same results again. The analyzer is (as one might hope) basically insensitive to these build flags.
Thanks. I ran under scan-build and got the same results. I am looking at these findings; but from the outset none of them seem relevant to the problems above.
The scan-build warnings are now fixed (see branch fix/440), but the failures remain. Proceeding.
Found (the location of) the problem. While scanning a block literal scalar, there is this code in `_scan_block()`:

https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parse_engine.def.hpp#L2085

Under `-O2` this check is failing for `lc.rem` consisting of `---`, which should not happen. Also, adding a print of `lc.rem` in the line just prior to this check will also cause the check to become correct.

Diving into `_is_doc_token()`, [we have originally](https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parse_engine.def.hpp#L89):
```cpp
inline bool _is_doc_token(csubstr s) // FAILS!
{
    if(s.len >= 3)
    {
        if(s.str[0] == '-')
            return _is_doc_begin_token(s);
        else if(s.str[0] == '.')
            return _is_doc_end_token(s);
    }
    return false;
}
```
If the calls are explicitly inlined, **the problem goes away**:
```cpp
inline bool _is_doc_token(csubstr s) // WORKS!
{
    if(s.len >= 3)
    {
        if(s.str[0] == '-')
            return (s.str[1] == '-' && s.str[2] == '-')
                && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
        else if(s.str[0] == '.')
            return (s.str[1] == '.' && s.str[2] == '.')
                && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
    }
    return false;
}
```
(note that a redundant check for `s.len >= 3` was removed in the inner conditions, but I tested keeping it as well, and this still succeeds). FWIW, the callee functions look like this:
```cpp
inline bool _is_doc_begin_token(csubstr s)
{
    RYML_ASSERT(s.begins_with('-'));
    RYML_ASSERT(!s.ends_with("\n"));
    RYML_ASSERT(!s.ends_with("\r"));
    return (s.len >= 3 && s.str[1] == '-' && s.str[2] == '-')
        && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
}

inline bool _is_doc_end_token(csubstr s)
{
    RYML_ASSERT(s.begins_with('.'));
    RYML_ASSERT(!s.ends_with("\n"));
    RYML_ASSERT(!s.ends_with("\r"));
    return (s.len >= 3 && s.str[1] == '.' && s.str[2] == '.')
        && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
}
```

---

So much for the trigger or, more precisely, the manifestation of the problem. What about the source of the problem? 

I'm not really seeing it.

Inside these functions, I see nothing that would trigger UB. My working theory is this: something may be  leading the optimizer to make erroneous assumptions about `lc.rem` (which is the string passed into the function). Although I'd like to, I'm on the fence whether I should spend time investigating this.
Looking further into `lc.rem`, it is set just above:

https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parse_engine.def.hpp#L2065

... and even when failing, the subsequent print correctly says `blck: peeking at [3]~~~---~~~`.

Anyway, [looking at that function](https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parser_state.hpp#L58):

```cpp
    void reset_with_next_line(substr buf, size_t offset)
    {
        RYML_ASSERT(offset <= buf.len);
        char const* C4_RESTRICT b = &buf[offset];
        char const* C4_RESTRICT e = b;
        // get the current line stripped of newline chars
        while(e < buf.end() && (*e != '\n' && *e != '\r'))
            ++e;
        RYML_ASSERT(e >= b);
        const substr stripped_ = buf.sub(offset, static_cast<size_t>(e - b));
        // advance pos to include the first line ending
        if(e != buf.end() && *e == '\r')
            ++e;
        if(e != buf.end() && *e == '\n')
            ++e;
        RYML_ASSERT(e >= b);
        const substr full_ = buf.sub(offset, static_cast<size_t>(e - b));
        reset(full_, stripped_);
    }
```
Maybe? In general, the function could be improved. In particular, looking at `e - b` which is signed, the optimizer could be having problems with the conversion to unsigned, and thereby taking (unwarranted) conclusions as to the length of the resulting substr. Maybe? Otherwise, I don't see much else going on in here.

To test, I've reinstated the problematic version of `_is_doc_token()` and I've changed to this:
```cpp
    void reset_with_next_line(substr buf, size_t offset)
    {
        RYML_ASSERT(offset <= buf.len);
        size_t e = offset;
        // get the current line stripped of newline chars
        while(e < buf.len && (buf.str[e] != '\n' && buf.str[e] != '\r'))
            ++e;
        RYML_ASSERT(e >= offset);
        const substr stripped_ = buf.range(offset, e);
        // advance pos to include the first line ending
        if(e < buf.len && buf.str[e] == '\r')
            ++e;
        if(e < buf.len && buf.str[e] == '\n')
            ++e;
        const substr full_ = buf.range(offset, e);
        reset(full_, stripped_);
    }
```
which is arguably cleaner. However, this has no impact on the failure; ie it manifests with both versions.

However, looking just below, there are comments mentioning gcc optimizer shenanigans with the same class:

https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parser_state.hpp#L89

So this may be related indeed.
I would conjecture that perhaps something somewhere is relying on undefined behavior, and as a result the optimizer ends up generating incorrect code – but an assertion happens to impede the optimization that reveals the problem. This would explain why the tests seem to pass when `NDEBUG` is not defined.
Thanks for the detailed repro, which is helpful.

I was able to reproduce the problem with gcc. But not with clang; didn't try with visual studio. 

FYI during the parser refactor I've [had similar trouble with the visual studio 32 bit optimizer](https://github.com/biojppm/rapidyaml/blob/31d8205d18d749f0410c42b1aff1c3ab3c14494c/src/c4/yml/parse_engine.def.hpp#L3194); none of the sanitizers or static analyzers were reporting any problem there or anywhere else. Like I noted there, I was only able to work around the problem, which I did by moving the problematic code to a separate function.

The outward signs are similar to that problem; if I enable the logs in the parser the failure goes away.

Also, using `-O3` makes the failure go away as well.

Fun times ahead.

> Thanks for the detailed repro, which is helpful.

You’re welcome. It took a few days of intermittently poking at this for me to get from “it breaks in my environment, but I can’t tell you why” to something useful enough to report.

> I was able to reproduce the problem with gcc (but not with clang; didn't try with visual studio).

I’m glad to hear it! At least we have consistency.

> Fun times ahead.

Yes, I always dread reporting bugs like this one because I know how hard they can be to investigate.

I’ll keep Fedora Rawhide at 0.6.0 for now. Please let me know if there’s anything I can do to help (other than actually finding the root cause!)
Well, the dread for these bugs is real, and pervasive. It is a laborious process to trace them to their source.

This command will reproduce all failures:
```
cd ~/proj/rapidyaml ;  \
( export BDIR=440_o2 ; \
  CXXFLAGS='-O2' cmake -S . -B $BDIR -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" -DRYML_BUILD_TESTS:BOOL=ON -DRYML_DBG:BOOL=OFF ; \
  pairs=( \
    "ryml-test-scalar_literal,--gtest_filter=*/YmlT*ix/7" \
    "ryml-test-scalar_folded,--gtest_filter=*/YmlT*ix/0" \
    "ryml-test-suite,W4TN-in_yaml /home/jpmag/proj/extern/yaml-test-suite/W4TN in.yaml --gtest_filter=_/unix_in_arena.1_compare_emitted_events_to_ref_events/0" \
  ) ; \
  for pair in "${pairs[@]}" ; do \
    IFS=, read tgt filt <<< "$pair"  ; \
    echo "$tgt -- $filt" ; \
    cmake --build $BDIR -j --verbose --target $tgt && \
    ./$BDIR/test/$tgt $filt ; \
  done \
) 
```

I'll keep recording relevant info here.
Well, the first two failures seem to be similar, and related to the parsing code: the parser seems to be slurping multiple docs into a single scalar. The error I linked to above is related to scalar filtering (ie happens when the scalar has already been slurped and is being filtered), and at first glance seems to be different from these two.

Here are the failed assertions for the first two failures:

## scalar_literal

```
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:162: Failure
Expected equality of these values:
  actualch.val()
    Which is: 'hello0
there
---
|
 hello1
 there
---
|
  hello2
  there
---
|
ciao3
qua
---
|
    ciao4
    qua
---
|
      ciao5
      qua
---
- |
 hello6
 there
- |
 ciao7
 qua
---
foo: |
 hello8
 there
bar: |
 ciao9
 qua
'
  expectedch.val
    Which is: 'hello0
there
'
Google Test trace:
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:158: 1
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:90: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:89: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:212: comparing: iteration based on the ref children
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:179: 0
/home/usr/proj/rapidyaml/test/test_lib/test_group.cpp:71: comparing parsed tree to ref tree
/home/usr/proj/rapidyaml/test/./test_lib/test_group.def.hpp:13: unix style
/home/usr/proj/rapidyaml/test/test_scalar_literal.cpp:1396: case
```

## scalar_folded

```
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:162: Failure
Expected equality of these values:
  actualch.val()
    Which is: 'hello there --- >
 hello
 there
--- >
  hello
  there
--- > ciao qua --- >
    ciao
    qua
--- >
      ciao
      qua
--- - >
 hello
 there
- >
 ciao
 qua
--- foo: >
 hello
 there
bar: >
 ciao
 qua
'
  expectedch.val
    Which is: 'hello there
'
Google Test trace:
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:158: 1
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:90: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:89: 0
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:212: comparing: iteration based on the ref children
/home/usr/proj/rapidyaml/test/test_lib/test_case_node.cpp:179: 0
/home/usr/proj/rapidyaml/test/test_lib/test_group.cpp:71: comparing parsed tree to ref tree
/home/usr/proj/rapidyaml/test/./test_lib/test_group.def.hpp:13: unix style
/home/usr/proj/rapidyaml/test/test_scalar_folded.cpp:1858: case
```

I’m able to get some output from Clang’s static analyzers like this:

```
$ CXXFLAGS='-O2 -g' CXX=clang++ scan-build cmake -S . -B build -DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG" -DRYML_BUILD_TESTS:BOOL=ON
$ scan-build cmake --build build -j16 --verbose
```

I am attaching the output directory as [scan-build-2024-06-21-132823-3551348-1.zip](https://github.com/user-attachments/files/15931091/scan-build-2024-06-21-132823-3551348-1.zip). It’s based on the latest `master`, 31d8205d18d749f0410c42b1aff1c3ab3c14494c. Hopefully there is something helpful in there.
Just for completeness, I tried the clang-analyzer/scan-build build with `-Og -g` and got exactly the same results. Then I tried omitting `-DCMAKE_CXX_FLAGS_RELEASE:STRING="-DNDEBUG"` and still got the same results again. The analyzer is (as one might hope) basically insensitive to these build flags.
Thanks. I ran under scan-build and got the same results. I am looking at these findings; but from the outset none of them seem relevant to the problems above.
The scan-build warnings are now fixed (see branch fix/440), but the failures remain. Proceeding.
Found (the location of) the problem. While scanning a block literal scalar, there is this code in `_scan_block()`:

https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parse_engine.def.hpp#L2085

Under `-O2` this check is failing for `lc.rem` consisting of `---`, which should not happen. Also, adding a print of `lc.rem` in the line just prior to this check will also cause the check to become correct.

Diving into `_is_doc_token()`, [we have originally](https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parse_engine.def.hpp#L89):
```cpp
inline bool _is_doc_token(csubstr s) // FAILS!
{
    if(s.len >= 3)
    {
        if(s.str[0] == '-')
            return _is_doc_begin_token(s);
        else if(s.str[0] == '.')
            return _is_doc_end_token(s);
    }
    return false;
}
```
If the calls are explicitly inlined, **the problem goes away**:
```cpp
inline bool _is_doc_token(csubstr s) // WORKS!
{
    if(s.len >= 3)
    {
        if(s.str[0] == '-')
            return (s.str[1] == '-' && s.str[2] == '-')
                && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
        else if(s.str[0] == '.')
            return (s.str[1] == '.' && s.str[2] == '.')
                && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
    }
    return false;
}
```
(note that a redundant check for `s.len >= 3` was removed in the inner conditions, but I tested keeping it as well, and this still succeeds). FWIW, the callee functions look like this:
```cpp
inline bool _is_doc_begin_token(csubstr s)
{
    RYML_ASSERT(s.begins_with('-'));
    RYML_ASSERT(!s.ends_with("\n"));
    RYML_ASSERT(!s.ends_with("\r"));
    return (s.len >= 3 && s.str[1] == '-' && s.str[2] == '-')
        && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
}

inline bool _is_doc_end_token(csubstr s)
{
    RYML_ASSERT(s.begins_with('.'));
    RYML_ASSERT(!s.ends_with("\n"));
    RYML_ASSERT(!s.ends_with("\r"));
    return (s.len >= 3 && s.str[1] == '.' && s.str[2] == '.')
        && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
}
```

---

So much for the trigger or, more precisely, the manifestation of the problem. What about the source of the problem? 

I'm not really seeing it.

Inside these functions, I see nothing that would trigger UB. My working theory is this: something may be  leading the optimizer to make erroneous assumptions about `lc.rem` (which is the string passed into the function). Although I'd like to, I'm on the fence whether I should spend time investigating this.
Looking further into `lc.rem`, it is set just above:

https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parse_engine.def.hpp#L2065

... and even when failing, the subsequent print correctly says `blck: peeking at [3]~~~---~~~`.

Anyway, [looking at that function](https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parser_state.hpp#L58):

```cpp
    void reset_with_next_line(substr buf, size_t offset)
    {
        RYML_ASSERT(offset <= buf.len);
        char const* C4_RESTRICT b = &buf[offset];
        char const* C4_RESTRICT e = b;
        // get the current line stripped of newline chars
        while(e < buf.end() && (*e != '\n' && *e != '\r'))
            ++e;
        RYML_ASSERT(e >= b);
        const substr stripped_ = buf.sub(offset, static_cast<size_t>(e - b));
        // advance pos to include the first line ending
        if(e != buf.end() && *e == '\r')
            ++e;
        if(e != buf.end() && *e == '\n')
            ++e;
        RYML_ASSERT(e >= b);
        const substr full_ = buf.sub(offset, static_cast<size_t>(e - b));
        reset(full_, stripped_);
    }
```
Maybe? In general, the function could be improved. In particular, looking at `e - b` which is signed, the optimizer could be having problems with the conversion to unsigned, and thereby taking (unwarranted) conclusions as to the length of the resulting substr. Maybe? Otherwise, I don't see much else going on in here.

To test, I've reinstated the problematic version of `_is_doc_token()` and I've changed to this:
```cpp
    void reset_with_next_line(substr buf, size_t offset)
    {
        RYML_ASSERT(offset <= buf.len);
        size_t e = offset;
        // get the current line stripped of newline chars
        while(e < buf.len && (buf.str[e] != '\n' && buf.str[e] != '\r'))
            ++e;
        RYML_ASSERT(e >= offset);
        const substr stripped_ = buf.range(offset, e);
        // advance pos to include the first line ending
        if(e < buf.len && buf.str[e] == '\r')
            ++e;
        if(e < buf.len && buf.str[e] == '\n')
            ++e;
        const substr full_ = buf.range(offset, e);
        reset(full_, stripped_);
    }
```
which is arguably cleaner. However, this has no impact on the failure; ie it manifests with both versions.

However, looking just below, there are comments mentioning gcc optimizer shenanigans with the same class:

https://github.com/biojppm/rapidyaml/blob/e83730943f5c81bec99145b6b286b1988e8e84c9/src/c4/yml/parser_state.hpp#L89

So this may be related indeed.

Fix patch:
diff --git a/.github/workflows/gcc.yml b/.github/workflows/gcc.yml
index 2a16edb2..e3660d92 100644
--- a/.github/workflows/gcc.yml
+++ b/.github/workflows/gcc.yml
@@ -71,6 +71,39 @@ jobs:
       - {name: shared32-run, run: source .github/setenv.sh && c4_run_test shared32}
       - {name: shared32-pack, run: source .github/setenv.sh && c4_package shared32}
 
+  #----------------------------------------------------------------------------
+  gcc_O2: # see https://github.com/biojppm/rapidyaml/issues/440
+    name: gcc_O2/${{matrix.cxx}}/c++${{matrix.std}}
+    continue-on-error: true
+    if: always()  # https://stackoverflow.com/questions/62045967/github-actions-is-there-a-way-to-continue-on-error-while-still-getting-correct
+    runs-on: ubuntu-latest
+    container: ghcr.io/biojppm/c4core/ubuntu22.04:latest  # use the docker image
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - {std: 11, gcc: 12 , bt: Release}
+    env: {STD: "${{matrix.std}}", CXX_: "${{matrix.gcc}}", BT: "${{matrix.bt}}", BITLINKS: "${{matrix.bitlinks}}", VG: "${{matrix.vg}}", SAN: "${{matrix.san}}", LINT: "${{matrix.lint}}", OS: "${{matrix.os}}"}
+    steps:
+      - {name: checkout, uses: actions/checkout@v3, with: {submodules: recursive}}
+      - run: git config --system --add safe.directory '*'  # needed for running in the docker image. see https://github.com/actions/checkout/issues/1169
+      - run: c4core-install g++-${{matrix.gcc}}
+      - {name: show info, run: source .github/setenv.sh && c4_show_info}
+      - name: configure
+        run: |
+          cmake -S . -B build_o2 \
+            -DCMAKE_CXX_COMPILER=g++-${{matrix.gcc}} \
+            -DCMAKE_C_COMPILER=gcc-${{matrix.gcc}} \
+            -DCMAKE_CXX_FLAGS_RELEASE:STRING="-O2 -g -DNDEBUG" \
+            -DRYML_BUILD_TESTS:BOOL=ON \
+            -DRYML_DBG:BOOL=OFF
+      - name: build
+        run: |
+          cmake --build build_o2 --target ryml-test-build -j --verbose
+      - name: run
+        run: |
+          cmake --build build_o2 --target ryml-test-run
+
   #----------------------------------------------------------------------------
   gcc_tabtokens:
     name: gcc_canary/${{matrix.cxx}}/c++${{matrix.std}}/${{matrix.bt}}
diff --git a/changelog/current.md b/changelog/current.md
index 2cde82e4..9f25ef5e 100644
--- a/changelog/current.md
+++ b/changelog/current.md
@@ -1,7 +1,17 @@
 ## Fixes
 
+- Fix [#440](https://github.com/biojppm/rapidyaml/issues/440): some tests failing with gcc -O2 (hypothetically due to undefined behavior)
+  - The fix was accomplished by refactoring some internal parser functions; see the comments in [#440](https://github.com/biojppm/rapidyaml/issues/440) for further details.
+  - Also, fix all warnings from `scan-build`.
+- Use malloc.h instead of alloca.h on MinGW ([PR#447](https://github.com/biojppm/rapidyaml/pull/447))
 - Fix [#442](https://github.com/biojppm/rapidyaml/issues/442) ([PR#443](https://github.com/biojppm/rapidyaml/pull/443)):
   - Ensure leading `+` is accepted when deserializing numbers.
   - Ensure numbers are not quoted by fixing the heuristics in `scalar_style_query_plain()` and `scalar_style_choose()`.
   - Add quickstart sample for overflow detection (only of integral types).
 - Parse engine: cleanup unused macros
+
+
+## Thanks
+
+- @toge
+- @musicinmybrain
diff --git a/ext/c4core b/ext/c4core
index f9e970ac..a95b20e8 160000
--- a/ext/c4core
+++ b/ext/c4core
@@ -1,1 +1,1 @@
-Subproject commit f9e970ac152034544d13dc3d9d109e17504274b1
+Subproject commit a95b20e8c02f4d8eb4425a85792d0f6f81007441
diff --git a/src/c4/yml/event_handler_tree.hpp b/src/c4/yml/event_handler_tree.hpp
index 6c08ecad..25369df8 100644
--- a/src/c4/yml/event_handler_tree.hpp
+++ b/src/c4/yml/event_handler_tree.hpp
@@ -104,11 +104,13 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
 
     void start_parse(const char* filename, detail::pfn_relocate_arena relocate_arena, void *relocate_arena_data)
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree != nullptr);
         this->_stack_start_parse(filename, relocate_arena, relocate_arena_data);
     }
 
     void finish_parse()
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree != nullptr);
         if(m_num_directives && !m_tree->is_stream(m_tree->root_id()))
             _RYML_CB_ERR_(m_stack.m_callbacks, "directives cannot be used without a document", {});
         this->_stack_finish_parse();
@@ -307,6 +309,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
 
     void add_sibling()
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         _RYML_CB_ASSERT(m_stack.m_callbacks, m_parent);
         _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree->has_children(m_parent->node_id));
         NodeData const* prev = m_tree->m_buf; // watchout against relocation of the tree nodes
@@ -445,6 +448,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
     void set_key_anchor(csubstr anchor)
     {
         _c4dbgpf("node[{}]: set key anchor: [{}]~~~{}~~~", m_curr->node_id, anchor.len, anchor);
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         if(C4_UNLIKELY(_has_any_(KEYREF)))
             _RYML_CB_ERR_(m_tree->callbacks(), "key cannot have both anchor and ref", m_curr->pos);
         _RYML_CB_ASSERT(m_tree->callbacks(), !anchor.begins_with('&'));
@@ -454,6 +458,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
     void set_val_anchor(csubstr anchor)
     {
         _c4dbgpf("node[{}]: set val anchor: [{}]~~~{}~~~", m_curr->node_id, anchor.len, anchor);
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         if(C4_UNLIKELY(_has_any_(VALREF)))
             _RYML_CB_ERR_(m_tree->callbacks(), "val cannot have both anchor and ref", m_curr->pos);
         _RYML_CB_ASSERT(m_tree->callbacks(), !anchor.begins_with('&'));
@@ -464,6 +469,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
     void set_key_ref(csubstr ref)
     {
         _c4dbgpf("node[{}]: set key ref: [{}]~~~{}~~~", m_curr->node_id, ref.len, ref);
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         if(C4_UNLIKELY(_has_any_(KEYANCH)))
             _RYML_CB_ERR_(m_tree->callbacks(), "key cannot have both anchor and ref", m_curr->pos);
         _RYML_CB_ASSERT(m_tree->callbacks(), ref.begins_with('*'));
@@ -474,6 +480,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
     void set_val_ref(csubstr ref)
     {
         _c4dbgpf("node[{}]: set val ref: [{}]~~~{}~~~", m_curr->node_id, ref.len, ref);
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         if(C4_UNLIKELY(_has_any_(VALANCH)))
             _RYML_CB_ERR_(m_tree->callbacks(), "val cannot have both anchor and ref", m_curr->pos);
         _RYML_CB_ASSERT(m_tree->callbacks(), ref.begins_with('*'));
@@ -541,6 +548,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
 
     substr alloc_arena(size_t len)
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         csubstr prev = m_tree->arena();
         substr out = m_tree->alloc_arena(len);
         substr curr = m_tree->arena();
@@ -551,6 +559,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
 
     substr alloc_arena(size_t len, substr *relocated)
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         csubstr prev = m_tree->arena();
         if(!prev.is_super(*relocated))
             return alloc_arena(len);
@@ -568,12 +577,13 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
     /** @cond dev */
     void _reset_parser_state(state* st, id_type parse_root, id_type node)
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         _set_state_(st, node);
         const NodeType type = m_tree->type(node);
         #ifdef RYML_DBG
         char flagbuf[80];
-        #endif
         _c4dbgpf("resetting state: initial flags={}", detail::_parser_flags_to_str(flagbuf, st->flags));
+        #endif
         if(type == NOTYPE)
         {
             _c4dbgpf("node[{}] is notype", node);
@@ -612,7 +622,9 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
             _c4dbgpf("node[{}] is doc", node);
             st->flags |= RDOC;
         }
+        #ifdef RYML_DBG
         _c4dbgpf("resetting state: final flags={}", detail::_parser_flags_to_str(flagbuf, st->flags));
+        #endif
     }
 
     /** push a new parent, add a child to the new parent, and set the
@@ -697,6 +709,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
     void _remove_speculative()
     {
         _c4dbgp("remove speculative node");
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         _RYML_CB_ASSERT(m_tree->callbacks(), m_tree->size() > 0);
         const id_type last_added = m_tree->size() - 1;
         if(m_tree->has_parent(last_added))
@@ -706,6 +719,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
 
     void _remove_speculative_with_parent()
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         _RYML_CB_ASSERT(m_tree->callbacks(), m_tree->size() > 0);
         const id_type last_added = m_tree->size() - 1;
         _RYML_CB_ASSERT(m_tree->callbacks(), m_tree->has_parent(last_added));
@@ -718,6 +732,7 @@ struct EventHandlerTree : public EventHandlerStack<EventHandlerTree, EventHandle
 
     C4_ALWAYS_INLINE void _save_loc()
     {
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_tree);
         _RYML_CB_ASSERT(m_tree->callbacks(), m_tree->_p(m_curr->node_id)->m_val.scalar.len == 0);
         m_tree->_p(m_curr->node_id)->m_val.scalar.str = m_curr->line_contents.rem.str;
     }
diff --git a/src/c4/yml/parse.cpp b/src/c4/yml/parse.cpp
index 144a6c4a..6651fa9f 100644
--- a/src/c4/yml/parse.cpp
+++ b/src/c4/yml/parse.cpp
@@ -28,7 +28,7 @@ inline void _reset_tree_handler(Parser *parser, Tree *t, id_type node_id)
     RYML_ASSERT(parser);
     RYML_ASSERT(t);
     if(!parser->m_evt_handler)
-        _RYML_CB_ERR(parser->m_evt_handler->m_stack.m_callbacks, "event handler is not set");
+        _RYML_CB_ERR(t->m_callbacks, "event handler is not set");
     parser->m_evt_handler->reset(t, node_id);
     RYML_ASSERT(parser->m_evt_handler->m_tree == t);
 }
diff --git a/src/c4/yml/parse_engine.def.hpp b/src/c4/yml/parse_engine.def.hpp
index c9929711..f400e10a 100644
--- a/src/c4/yml/parse_engine.def.hpp
+++ b/src/c4/yml/parse_engine.def.hpp
@@ -86,14 +86,33 @@ inline bool _is_doc_end_token(csubstr s)
         && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
 }
 
-inline bool _is_doc_token(csubstr s)
+inline bool _is_doc_token(csubstr s) noexcept
 {
+    //
+    // NOTE: this function was failing under some scenarios when
+    // compiled with gcc -O2 (but not -O3 or -O1 or -O0), likely
+    // related to optimizer assumptions on the input string and
+    // possibly caused from UB around assignment to that string (the
+    // call site was in _scan_block()). For more details see:
+    //
+    // https://github.com/biojppm/rapidyaml/issues/440
+    //
+    // The current version does not suffer this problem, but it may
+    // appear again.
+    //
     if(s.len >= 3)
     {
-        if(s.str[0] == '-')
-            return _is_doc_begin_token(s);
-        else if(s.str[0] == '.')
-            return _is_doc_end_token(s);
+        switch(s.str[0])
+        {
+        case '-':
+            //return _is_doc_begin_token(s); // this was failing with gcc -O2
+            return (s.str[1] == '-' && s.str[2] == '-')
+                && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
+        case '.':
+            //return _is_doc_end_token(s); // this was failing with gcc -O2
+            return (s.str[1] == '.' && s.str[2] == '.')
+                && (s.len == 3 || (s.str[3] == ' ' _RYML_WITH_TAB_TOKENS(|| s.str[3] == '\t')));
+        }
     }
     return false;
 }
@@ -2026,7 +2045,7 @@ void ParseEngine<EventHandler>::_scan_block(ScannedBlock *C4_RESTRICT sb, size_t
         digits = t.left_of(t.first_not_of("0123456789"));
         if( ! digits.empty())
         {
-            if(digits.len > 1)
+            if(C4_UNLIKELY(digits.len > 1))
                 _c4err("parse error: invalid indentation");
             _c4dbgpf("blck: parse indentation digits: [{}]~~~{}~~~", digits.len, digits);
             if(C4_UNLIKELY( ! c4::atou(digits, &indentation)))
@@ -2067,8 +2086,9 @@ void ParseEngine<EventHandler>::_scan_block(ScannedBlock *C4_RESTRICT sb, size_t
         // evaluate termination conditions
         if(indentation != npos)
         {
+            _c4dbgpf("blck: indentation={}", indentation);
             // stop when the line is deindented and not empty
-            if(lc.indentation < indentation && ( ! lc.rem.trim(" \t\r\n").empty()))
+            if(lc.indentation < indentation && ( ! lc.rem.trim(" \t").empty()))
             {
                 if(raw_block.len)
                 {
@@ -2082,6 +2102,7 @@ void ParseEngine<EventHandler>::_scan_block(ScannedBlock *C4_RESTRICT sb, size_t
             }
             else if(indentation == 0)
             {
+                _c4dbgpf("blck: noindent. lc.rem=[{}]~~~{}~~~", lc.rem.len, lc.rem);
                 if(_is_doc_token(lc.rem))
                 {
                     _c4dbgp("blck: stop. indentation=0 and doc ended");
@@ -3222,7 +3243,7 @@ void ParseEngine<EventHandler>::_filter_block_folded_newlines(FilterProcessor &C
             // In the end, moving this block to a separate function
             // was the only way to bury the problem. But it may
             // resurface again, as The Undead, rising to from the
-            // grave to haunt us with his terrible
+            // grave to haunt us with his terrible presence.
             //
             // We may have to revisit this. With a stake, and lots of
             // garlic.
diff --git a/src/c4/yml/parser_state.hpp b/src/c4/yml/parser_state.hpp
index b1bcfdd6..1977cbf5 100644
--- a/src/c4/yml/parser_state.hpp
+++ b/src/c4/yml/parser_state.hpp
@@ -58,30 +58,27 @@ struct LineContents
     void reset_with_next_line(substr buf, size_t offset)
     {
         RYML_ASSERT(offset <= buf.len);
-        char const* C4_RESTRICT b = &buf[offset];
-        char const* C4_RESTRICT e = b;
+        size_t e = offset;
         // get the current line stripped of newline chars
-        while(e < buf.end() && (*e != '\n' && *e != '\r'))
+        while(e < buf.len && (buf.str[e] != '\n' && buf.str[e] != '\r'))
             ++e;
-        RYML_ASSERT(e >= b);
-        const substr stripped_ = buf.sub(offset, static_cast<size_t>(e - b));
+        RYML_ASSERT(e >= offset);
+        const substr stripped_ = buf.range(offset, e);
         // advance pos to include the first line ending
-        if(e != buf.end() && *e == '\r')
+        if(e < buf.len && buf.str[e] == '\r')
             ++e;
-        if(e != buf.end() && *e == '\n')
+        if(e < buf.len && buf.str[e] == '\n')
             ++e;
-        RYML_ASSERT(e >= b);
-        const substr full_ = buf.sub(offset, static_cast<size_t>(e - b));
+        const substr full_ = buf.range(offset, e);
         reset(full_, stripped_);
     }
 
     void reset(substr full_, substr stripped_)
     {
+        rem = stripped_;
+        indentation = stripped_.first_not_of(' ');  // find the first column where the character is not a space
         full = full_;
         stripped = stripped_;
-        rem = stripped_;
-        // find the first column where the character is not a space
-        indentation = stripped.first_not_of(' ');
     }
 
     C4_ALWAYS_INLINE size_t current_col() const RYML_NOEXCEPT
diff --git a/src/c4/yml/reference_resolver.cpp b/src/c4/yml/reference_resolver.cpp
index a4ddfba8..392699a0 100644
--- a/src/c4/yml/reference_resolver.cpp
+++ b/src/c4/yml/reference_resolver.cpp
@@ -208,16 +208,10 @@ void ReferenceResolver::resolve(Tree *t_)
             _c4dbgpf("ref {} has parent: {}", i, refdata.parent_ref);
             _RYML_CB_ASSERT(m_tree->m_callbacks, m_tree->is_seq(refdata.parent_ref));
             const id_type p = m_tree->parent(refdata.parent_ref);
-            id_type after;
-            if(prev_parent_ref != refdata.parent_ref)
-            {
-                after = refdata.parent_ref;//prev_sibling(rd.parent_ref_sibling);
-                prev_parent_ref_after = after;
-            }
-            else
-            {
-                after = prev_parent_ref_after;
-            }
+            const id_type after = (prev_parent_ref != refdata.parent_ref) ?
+                refdata.parent_ref//prev_sibling(rd.parent_ref_sibling)
+                :
+                prev_parent_ref_after;
             prev_parent_ref = refdata.parent_ref;
             prev_parent_ref_after = m_tree->duplicate_children_no_rep(refdata.target, p, after);
             m_tree->remove(refdata.node);


Test patch:
diff --git a/test/test_noderef.cpp b/test/test_noderef.cpp
index 5ecffb10..da919b3e 100644
--- a/test/test_noderef.cpp
+++ b/test/test_noderef.cpp
@@ -1118,14 +1118,12 @@ TEST(NodeRef, vsConstNodeRef)
     // mseq = seq; // deliberate compilation error
     seq = mseq; // ok
     {
-        NodeData *nd = mseq.get();
-        // nd = seq.get(); // deliberate compile error
-        C4_UNUSED(nd);
+        (void)mseq.get();
+        //(void)seq.get(); // deliberate compile error
     }
     {
-        NodeData const* nd = seq.get();
-        nd = seq.get(); // ok
-        C4_UNUSED(nd);
+        (void)seq.get();
+        (void)seq.get(); // ok
     }
     test_invariants(t);
 }
diff --git a/test/test_serialize.cpp b/test/test_serialize.cpp
index 8492e06f..fe2dfdcd 100644
--- a/test/test_serialize.cpp
+++ b/test/test_serialize.cpp
@@ -565,11 +565,11 @@ void test_deserialize_trailing_434(csubstr yaml, csubstr val, csubstr first, dou
     ConstNodeRef cnode = tree["val"];
     NodeRef node = tree["val"];
     {
-        int value;
+        int value = {};
         EXPECT_FALSE(read(cnode, &value));
     }
     {
-        int value;
+        int value = {};
         EXPECT_FALSE(read(node, &value));
     }
     ExpectError::do_check(&tree, [&]{
@@ -582,22 +582,22 @@ void test_deserialize_trailing_434(csubstr yaml, csubstr val, csubstr first, dou
     });
     float fval = (float)dval;
     {
-        float value;
+        float value = {};
         EXPECT_TRUE(read(cnode, &value));
         EXPECT_EQ(memcmp(&value, &fval, sizeof(fval)), 0);
     }
     {
-        float value;
+        float value = {};
         EXPECT_TRUE(read(node, &value));
         EXPECT_EQ(memcmp(&value, &fval, sizeof(fval)), 0);
     }
     {
-        double value;
+        double value = {};
         EXPECT_TRUE(read(cnode, &value));
         EXPECT_EQ(memcmp(&value, &dval, sizeof(dval)), 0);
     }
     {
-        double value;
+        double value = {};
         EXPECT_TRUE(read(node, &value));
         EXPECT_EQ(memcmp(&value, &dval, sizeof(dval)), 0);
     }
diff --git a/test/test_tree.cpp b/test/test_tree.cpp
index 5e75cf3c..7ebb23d5 100644
--- a/test/test_tree.cpp
+++ b/test/test_tree.cpp
@@ -719,9 +719,8 @@ TEST(Tree, move_ctor)
     test_compare(save, src);
     {
         Tree dst(std::move(src));
-        EXPECT_EQ(src.empty(), true);
-        EXPECT_EQ(src.size(), 0u);
-        EXPECT_EQ(src.arena().empty(), true);
+        EXPECT_EQ(src.m_size, 0u);
+        EXPECT_EQ(src.m_arena_pos, 0u);
         EXPECT_EQ(dst.size(), save.size());
         EXPECT_EQ(dst.arena(), save.arena());
         test_invariants(src);
@@ -789,10 +788,9 @@ TEST(Tree, move_assign_same_callbacks)
         EXPECT_NE(dst.size(), 0u);
         EXPECT_NE(dst.arena().empty(), true);
         dst = std::move(src);
-        EXPECT_EQ(src.empty(), true);
-        EXPECT_EQ(src.size(), 0u);
-        EXPECT_EQ(src.arena().empty(), true);
-        EXPECT_EQ(src.callbacks(), cbt.callbacks());
+        EXPECT_EQ(src.m_size, 0u);
+        EXPECT_EQ(src.m_arena_pos, 0u);
+        EXPECT_EQ(src.m_callbacks, cbt.callbacks());
         EXPECT_EQ(dst.size(), save.size());
         EXPECT_EQ(dst.arena(), save.arena());
         EXPECT_EQ(dst.callbacks(), save.callbacks());
@@ -820,10 +818,9 @@ TEST(Tree, move_assign_diff_callbacks)
         EXPECT_NE(dst.arena().empty(), true);
         EXPECT_EQ(dst.callbacks(), cbdst.callbacks());
         dst = std::move(src);
-        EXPECT_EQ(src.empty(), true);
-        EXPECT_EQ(src.size(), 0u);
-        EXPECT_EQ(src.arena().empty(), true);
-        EXPECT_EQ(src.callbacks(), cbsrc.callbacks());
+        EXPECT_EQ(src.m_size, 0u);
+        EXPECT_EQ(src.m_arena_pos, 0u);
+        EXPECT_EQ(src.m_callbacks, cbsrc.callbacks());
         EXPECT_EQ(dst.size(), save.size());
         EXPECT_EQ(dst.arena(), save.arena());
         EXPECT_NE(dst.callbacks(), cbdst.callbacks());
@@ -3714,16 +3711,16 @@ TEST(Tree, lookup_path_or_modify)
         EXPECT_EQ(t["newmap2"]["newseq2"][2]["newmap2"]["newseq2"][2].val(), nullptr);
         EXPECT_EQ(t["newmap2"]["newseq2"][2]["newmap2"]["newseq2"][3].is_map(), true);
         EXPECT_EQ(t["newmap2"]["newseq2"][2]["newmap2"]["newseq2"][3]["first2"].val(), "y");
-        sz2 = t.lookup_path_or_modify("z", "newmap2.newseq2[2].newmap2.newseq2[3].second2");
+        (void)t.lookup_path_or_modify("z", "newmap2.newseq2[2].newmap2.newseq2[3].second2");
         EXPECT_EQ  (t["newmap2"]["newseq2"][2]["newmap2"]["newseq2"][3]["second2"].val(), "z");
 
-        sz = t.lookup_path_or_modify("foo", "newmap.newseq1[1]");
+        (void)t.lookup_path_or_modify("foo", "newmap.newseq1[1]");
         EXPECT_EQ(t["newmap"].is_map(), true);
         EXPECT_EQ(t["newmap"]["newseq1"].is_seq(), true);
         EXPECT_EQ(t["newmap"]["newseq1"].num_children(), 2u);
         EXPECT_EQ(t["newmap"]["newseq1"][0].val(), nullptr);
         EXPECT_EQ(t["newmap"]["newseq1"][1].val(), "foo");
-        sz = t.lookup_path_or_modify("bar", "newmap.newseq1[2][1]");
+        (void)t.lookup_path_or_modify("bar", "newmap.newseq1[2][1]");
         EXPECT_EQ(t["newmap"]["newseq1"].is_seq(), true);
         EXPECT_EQ(t["newmap"]["newseq1"].num_children(), 3u);
         EXPECT_EQ(t["newmap"]["newseq1"][0].val(), nullptr);
@@ -3732,12 +3729,12 @@ TEST(Tree, lookup_path_or_modify)
         EXPECT_EQ(t["newmap"]["newseq1"][2].num_children(), 2u);
         EXPECT_EQ(t["newmap"]["newseq1"][2][0].val(), nullptr);
         EXPECT_EQ(t["newmap"]["newseq1"][2][1].val(), "bar");
-        sz = t.lookup_path_or_modify("Foo?"   , "newmap.newseq1[0]");
-        sz = t.lookup_path_or_modify("Bar?"   , "newmap.newseq1[2][0]");
-        sz = t.lookup_path_or_modify("happy"  , "newmap.newseq1[2][2][3]");
-        sz = t.lookup_path_or_modify("trigger", "newmap.newseq1[2][2][2]");
-        sz = t.lookup_path_or_modify("Arnold" , "newmap.newseq1[2][2][0]");
-        sz = t.lookup_path_or_modify("is"     , "newmap.newseq1[2][2][1]");
+        (void)t.lookup_path_or_modify("Foo?"   , "newmap.newseq1[0]");
+        (void)t.lookup_path_or_modify("Bar?"   , "newmap.newseq1[2][0]");
+        (void)t.lookup_path_or_modify("happy"  , "newmap.newseq1[2][2][3]");
+        (void)t.lookup_path_or_modify("trigger", "newmap.newseq1[2][2][2]");
+        (void)t.lookup_path_or_modify("Arnold" , "newmap.newseq1[2][2][0]");
+        (void)t.lookup_path_or_modify("is"     , "newmap.newseq1[2][2][1]");
         EXPECT_EQ(t["newmap"]["newseq1"].is_seq(), true);
         EXPECT_EQ(t["newmap"]["newseq1"].num_children(), 3u);
         EXPECT_EQ(t["newmap"]["newseq1"][0].val(), "Foo?");