Repository: tafia/quick-xml (rust)

Issue:
Text events are not emitted when text node ends exactly at BufferedReader buffer border
Perhaps the title isn't quite clear here. As a concrete example, the document written by this python script:

```py
open('test.xml', 'w').write('<asdf>' + 'a' * (8192 - 6) + '</asdf>')
```

(i.e., a document with a single tag, whose contents end at offset 8192) will cause quick-xml to emit only a Start and End event, with no Text event in the middle. This can be verified with the following rust program, which simply dumps all received events to stderr:

```rs
use quick_xml::events::Event;
use quick_xml::reader::Reader;

fn main() {
    let mut reader = Reader::from_file("test.xml").unwrap();
    let mut buf = Vec::new();
    loop {
        let ev = reader.read_event_into(&mut buf);
        if matches!(ev, Ok(Event::Eof)) { break; }
        dbg!(&ev);
    }
}
```

This prints:

```
[src/main.rs:10:9] &ev = Ok(
    Start(
        BytesStart { buf: Borrowed("asdf"), name_len: 4 },
    ),
)
[src/main.rs:10:9] &ev = Ok(
    End(
        BytesEnd { name: Borrowed("asdf") },
    ),
)
```

---

The root cause seems to be [in the `read_text` of `buffered_reader.rs`](https://github.com/tafia/quick-xml/blob/80f0e7cc4cc3dedf416b9f4f4efd35d89b67cd8c/src/reader/buffered_reader.rs#L53). In this case, reading the text data requires two loop iterations. On the first iteration, `<` is not found in the data, so the entire data is pushed onto `buf` and the loop continues. On the second iteration, `<` is found at position 0, which triggers the special case in the code that returns `ReadTextResult::Markup` instead of `ReadTextResult::UpToMarkup`, and does not indicate that text data was also present.

I think the right fix here would be to only check the zero-position case on the first iteration of the loop.

Pull request:
Fix missing text event in some circumstances
Specially handle `<` in the first position of read data only after first `fill_buf()` call

Fixes #774

Patch:
diff --git a/Changelog.md b/Changelog.md
index cbcb7fe..684999e 100644
--- a/Changelog.md
+++ b/Changelog.md
@@ -27,6 +27,8 @@
   - `TooLongDecimal`
   - `TooLongHexadecimal`
 - [#771]: Fixed `Attribute::unescape_value` which does not unescape predefined values since 0.32.0.
+- [#774]: Fixed regression since 0.33.0: `Text` event may be skipped in `read_event_into()`
+  and `read_event_into_async()` in some circumstances.
 
 ### Misc Changes
 
@@ -42,6 +44,7 @@
 [#771]: https://github.com/tafia/quick-xml/pull/771
 [#772]: https://github.com/tafia/quick-xml/pull/772
 [#773]: https://github.com/tafia/quick-xml/pull/773
+[#774]: https://github.com/tafia/quick-xml/issues/774
 
 
 ## 0.34.0 -- 2024-06-25
diff --git a/src/reader/buffered_reader.rs b/src/reader/buffered_reader.rs
index 548fea9..cd1e5f4 100644
--- a/src/reader/buffered_reader.rs
+++ b/src/reader/buffered_reader.rs
@@ -69,7 +69,9 @@ macro_rules! impl_buffered_source {
                 };
 
                 match memchr::memchr(b'<', available) {
-                    Some(0) => {
+                    // Special handling is needed only on the first iteration.
+                    // On next iterations we already read something and should emit Text event
+                    Some(0) if read == 0 => {
                         self $(.$reader)? .consume(1);
                         *position += 1;
                         return ReadTextResult::Markup(buf);


Test patch:
diff --git a/tests/async-tokio.rs b/tests/async-tokio.rs
index 3c267d9..b525dfb 100644
--- a/tests/async-tokio.rs
+++ b/tests/async-tokio.rs
@@ -1,9 +1,10 @@
 use std::iter;
 
 use pretty_assertions::assert_eq;
-use quick_xml::events::{BytesStart, Event::*};
+use quick_xml::events::{BytesEnd, BytesStart, BytesText, Event::*};
 use quick_xml::name::QName;
 use quick_xml::reader::Reader;
+use tokio::io::BufReader;
 
 // Import `small_buffers_tests!`
 #[macro_use]
@@ -148,3 +149,29 @@ async fn issue751() {
         }
     }
 }
+
+/// Regression test for https://github.com/tafia/quick-xml/issues/774
+///
+/// Capacity of the buffer selected in that way, that "text" will be read into
+/// one internal buffer of `BufReader` in one `fill_buf()` call and `<` of the
+/// closing tag in the next call.
+#[tokio::test]
+async fn issue774() {
+    let xml = BufReader::with_capacity(9, b"<tag>text</tag>" as &[u8]);
+    //                                      ^0       ^9
+    let mut reader = Reader::from_reader(xml);
+    let mut buf = Vec::new();
+
+    assert_eq!(
+        reader.read_event_into_async(&mut buf).await.unwrap(),
+        Start(BytesStart::new("tag"))
+    );
+    assert_eq!(
+        reader.read_event_into_async(&mut buf).await.unwrap(),
+        Text(BytesText::new("text"))
+    );
+    assert_eq!(
+        reader.read_event_into_async(&mut buf).await.unwrap(),
+        End(BytesEnd::new("tag"))
+    );
+}
diff --git a/tests/issues.rs b/tests/issues.rs
index 163a17b..e4a7569 100644
--- a/tests/issues.rs
+++ b/tests/issues.rs
@@ -2,11 +2,12 @@
 //!
 //! Name each module / test as `issue<GH number>` and keep sorted by issue number
 
+use std::io::BufReader;
 use std::iter;
 use std::sync::mpsc;
 
 use quick_xml::errors::{Error, IllFormedError, SyntaxError};
-use quick_xml::events::{BytesDecl, BytesStart, BytesText, Event};
+use quick_xml::events::{BytesDecl, BytesEnd, BytesStart, BytesText, Event};
 use quick_xml::name::QName;
 use quick_xml::reader::Reader;
 
@@ -337,3 +338,29 @@ fn issue751() {
         }
     }
 }
+
+/// Regression test for https://github.com/tafia/quick-xml/issues/774
+///
+/// Capacity of the buffer selected in that way, that "text" will be read into
+/// one internal buffer of `BufReader` in one `fill_buf()` call and `<` of the
+/// closing tag in the next call.
+#[test]
+fn issue774() {
+    let xml = BufReader::with_capacity(9, b"<tag>text</tag>" as &[u8]);
+    //                                      ^0       ^9
+    let mut reader = Reader::from_reader(xml);
+    let mut buf = Vec::new();
+
+    assert_eq!(
+        reader.read_event_into(&mut buf).unwrap(),
+        Event::Start(BytesStart::new("tag"))
+    );
+    assert_eq!(
+        reader.read_event_into(&mut buf).unwrap(),
+        Event::Text(BytesText::new("text"))
+    );
+    assert_eq!(
+        reader.read_event_into(&mut buf).unwrap(),
+        Event::End(BytesEnd::new("tag"))
+    );
+}