Repository: mozilla/bleach

Issue:
Open bracket '<' still cleaned up without closing bracket
**Describe the bug**

Thanks for the fix provided https://github.com/mozilla/bleach/issues/705!
I think I found regression after that fix

 - Python Version: [e.g. 3.9.6]
 - Bleach Version: [e.g. 6.1.0]

**To Reproduce**

Steps to reproduce the behavior:

```python
# Working!
In [5]: bleach.clean("<test abc")
Out[5]: '&lt;test abc'
```

```python
# Doesn't work (because of duplicated words?)
In [6]: bleach.clean("<test abc abc")
Out[6]: ''
```

```python
# However this work
In [12]: bleach.clean("<test abc abd")
Out[12]: '&lt;test abc abd'
```

```python
# Doesn't work (with space in the end)
In [7]: bleach.clean("<test abc ")
Out[7]: ''
```

```python
# Doesn't work (with space in the end)
In [8]: bleach.clean("asd<test abc ")
Out[8]: 'asd'
```

```python
# However this work
In [9]: bleach.clean("asd<test abc asd")
Out[9]: 'asd&lt;test abc asd'
```

**Expected behavior**

```python
# Doesn't work (because of duplicated words?)
In [6]: bleach.clean("<test abc abc")
Out[6]: '&lt;test abc abc'
```

```python
# Doesn't work (with space in the end)
In [7]: bleach.clean("<test abc ")
Out[7]: '&lt;test abc '
```

```python
# Doesn't work (with space in the end)
In [8]: bleach.clean("asd<test abc ")
Out[8]: 'asd&lt;test abc '
```

**Additional context**

Add any other context about the problem here.


Discussion:

Patch:
diff --git a/bleach/html5lib_shim.py b/bleach/html5lib_shim.py
index ca1cc8c..f083db7 100644
--- a/bleach/html5lib_shim.py
+++ b/bleach/html5lib_shim.py
@@ -396,16 +396,25 @@ class BleachHTMLTokenizer(HTMLTokenizer):
                 # name that abruptly ends, but we should treat that like
                 # character data
                 yield {"type": TAG_TOKEN_TYPE_CHARACTERS, "data": self.stream.get_tag()}
+
             elif last_error_token["data"] in (
+                "duplicate-attribute",
                 "eof-in-attribute-name",
                 "eof-in-attribute-value-no-quotes",
+                "expected-end-of-tag-but-got-eof",
             ):
                 # Handle the case where the text being parsed ends with <
-                # followed by a series of characters and then space and then
-                # more characters. It's treated as a tag name followed by an
+                # followed by characters and then space and then:
+                #
+                # * more characters
+                # * more characters repeated with a space between (e.g. "abc abc")
+                # * more characters and then a space and then an EOF (e.g. "abc def ")
+                #
+                # These cases are treated as a tag name followed by an
                 # attribute that abruptly ends, but we should treat that like
-                # character data.
+                # character data instead.
                 yield {"type": TAG_TOKEN_TYPE_CHARACTERS, "data": self.stream.get_tag()}
+
             else:
                 yield last_error_token
 


Test patch:
diff --git a/tests/test_clean.py b/tests/test_clean.py
index dc129d0..da4082a 100644
--- a/tests/test_clean.py
+++ b/tests/test_clean.py
@@ -167,6 +167,10 @@ def test_bare_entities_get_escaped_correctly(text, expected):
         ("<some thing", "&lt;some thing"),
         # this is an eof-in-attribute-value-no-quotes parser error
         ("<some thing=foo", "&lt;some thing=foo"),
+        # this is a duplicate-attribute parser error
+        ("<some thing thing", "&lt;some thing thing"),
+        # this is an expected-end-of-tag-but-got-eof parser error
+        ("<some thing thing2 ", "&lt;some thing thing2 "),
     ],
 )
 def test_lessthan_escaping(text, expected):