internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: tornadoweb/tornado

Issue:
Minor bug in WSGIContainer docstring.
https://github.com/tornadoweb/tornado/blob/2047e7ae3c825bf52dad10cc8402d09e11091bc1/tornado/wsgi.py#L76

This line (in a docstring showing how to use WSGIContainer) is wrong.

It should say:

             return [b"Hello world!\n"] 

... because WSGI requires a sequence of bytestrings in modern python. Without this change, the example  fails with:

    TypeError: sequence item 0: expected a bytes-like object, str found

in python 3.9 (and probably any python 3).


Discussion:
Go ahead and open a pull request, it will likely be accepted.

Resolving patch:
diff --git a/tornado/wsgi.py b/tornado/wsgi.py
--- a/tornado/wsgi.py
+++ b/tornado/wsgi.py
@@ -73,7 +73,7 @@ def simple_app(environ, start_response):
             status = "200 OK"
             response_headers = [("Content-type", "text/plain")]
             start_response(status, response_headers)
-            return ["Hello world!\n"]
+            return [b"Hello world!\n"]
 
         container = tornado.wsgi.WSGIContainer(simple_app)
         http_server = tornado.httpserver.HTTPServer(container)


Test patch:
Repository: tornadoweb/tornado

Issue:
AttributeError: 'NoneType' object has no attribute 'write_message'
`tornado.websocket.WebSocketClientConnection.write_message` documentation says:

> If the stream is closed, raises `WebSocketClosedError`.

But after the stream is closed  `self.protocol` turns to None, and the line `self.protocol.write_message` raises `AttributeError`


Discussion:

Resolving patch:
diff --git a/tornado/websocket.py b/tornado/websocket.py
--- a/tornado/websocket.py
+++ b/tornado/websocket.py
@@ -1501,6 +1501,8 @@ def write_message(
            Exception raised on a closed stream changed from `.StreamClosedError`
            to `WebSocketClosedError`.
         """
+        if self.protocol is None:
+            raise WebSocketClosedError("Client connection has been closed")
         return self.protocol.write_message(message, binary=binary)
 
     def read_message(


Test patch:
diff --git a/tornado/test/websocket_test.py b/tornado/test/websocket_test.py
--- a/tornado/test/websocket_test.py
+++ b/tornado/test/websocket_test.py
@@ -339,6 +339,13 @@ def test_unicode_message(self):
         response = yield ws.read_message()
         self.assertEqual(response, u"hello \u00e9")
 
+    @gen_test
+    def test_error_in_closed_client_write_message(self):
+        ws = yield self.ws_connect("/echo")
+        ws.close()
+        with self.assertRaises(WebSocketClosedError):
+            ws.write_message(u"hello \u00e9")
+
     @gen_test
     def test_render_message(self):
         ws = yield self.ws_connect("/render")
Repository: tornadoweb/tornado

Issue:
Replace gen.TimeoutError with asyncio.TimeoutError
As applications increasingly use Tornado within Asyncio frameworks I find it challenging to figure out when I should be watching for `gen.TimeoutError`s and when I should be watching out for `asyncio.TimeoutErrors`.  I wonder if it is possible to unify the two

```python
# gen.py

TimeoutError = asyncio.TimeoutError
```

This would simplify things a bit downstream


Discussion:

Resolving patch:
diff --git a/tornado/gen.py b/tornado/gen.py
--- a/tornado/gen.py
+++ b/tornado/gen.py
@@ -606,6 +606,9 @@ def with_timeout(
     .. versionchanged:: 6.0.3
        ``asyncio.CancelledError`` is now always considered "quiet".
 
+    .. versionchanged:: 6.2
+       ``tornado.util.TimeoutError`` is now an alias to ``asyncio.TimeoutError``.
+
     """
     # It's tempting to optimize this by cancelling the input future on timeout
     # instead of creating a new one, but A) we can't know if we are the only
diff --git a/tornado/ioloop.py b/tornado/ioloop.py
--- a/tornado/ioloop.py
+++ b/tornado/ioloop.py
@@ -468,7 +468,7 @@ def run_sync(self, func: Callable, timeout: Optional[float] = None) -> Any:
 
         The keyword-only argument ``timeout`` may be used to set
         a maximum duration for the function.  If the timeout expires,
-        a `tornado.util.TimeoutError` is raised.
+        a `asyncio.TimeoutError` is raised.
 
         This method is useful to allow asynchronous calls in a
         ``main()`` function::
@@ -485,6 +485,8 @@ async def main():
         .. versionchanged:: 5.0
            If a timeout occurs, the ``func`` coroutine will be cancelled.
 
+        .. versionchanged:: 6.2
+           ``tornado.util.TimeoutError`` is now an alias to ``asyncio.TimeoutError``.
         """
         future_cell = [None]  # type: List[Optional[Future]]
 
diff --git a/tornado/util.py b/tornado/util.py
--- a/tornado/util.py
+++ b/tornado/util.py
@@ -11,6 +11,7 @@
 """
 
 import array
+import asyncio
 import atexit
 from inspect import getfullargspec
 import os
@@ -63,14 +64,9 @@ def is_finalizing() -> bool:
     is_finalizing = _get_emulated_is_finalizing()
 
 
-class TimeoutError(Exception):
-    """Exception raised by `.with_timeout` and `.IOLoop.run_sync`.
-
-    .. versionchanged:: 5.0:
-       Unified ``tornado.gen.TimeoutError`` and
-       ``tornado.ioloop.TimeoutError`` as ``tornado.util.TimeoutError``.
-       Both former names remain as aliases.
-    """
+# versionchanged:: 6.2
+# no longer our own TimeoutError, use standard asyncio class
+TimeoutError = asyncio.TimeoutError
 
 
 class ObjectDict(Dict[str, Any]):


Test patch:
diff --git a/tornado/test/import_test.py b/tornado/test/import_test.py
--- a/tornado/test/import_test.py
+++ b/tornado/test/import_test.py
@@ -64,3 +64,4 @@ def test_import_aliases(self):
 
         self.assertIs(tornado.ioloop.TimeoutError, tornado.util.TimeoutError)
         self.assertIs(tornado.gen.TimeoutError, tornado.util.TimeoutError)
+        self.assertIs(tornado.util.TimeoutError, asyncio.TimeoutError)
Repository: tornadoweb/tornado

Issue:
Tornado 6.2 release readiness
I'm creating this issue to collect feedback on the 6.2 betas. For the folks who have tried them, do you think the release is ready to go or are there still more changes to be made?

Tagging @minrk and @graingert as authors of relevant PRs, although I'd welcome feedback from anyone interested in this release. 


Discussion:
Jupyter uses it pretty extensively, and the only issues I believe we've had have been some strict checks for deprecation warnings. We've handled those through some calls to `IOLoop(make_current=False)` and some other ignoring deprecations to wait for larger refactors for now.

Our pattern of

1. instantiate and configure things (our 'initialize' stage), and
2. start the event loop (our 'start' stage)

is mostly handled by migrating to `IOLoop(make_current=False)`.

Where we have seen some hiccups is in when steps in `initialize` assumed `IOLoop.current()` could be used, which is no longer safe. We'll need to deal with these by either moving them into `start` or adding an async aspect to the initialize stage. Running as much of initialize as possibile within a `loop.run_sync` would solve every issue we have, aside from the fact that this currently _includes_ setting up the loop itself. One example we found is e.g. `PeriodicCallback` removed the `ioloop` arg some time ago, instead assuming `current` (made sense at the time). But this now means there is no longer _any_ way to instantiate PeriodicCallback outside a running loop. Not the biggest deal for us, just needs some reshuffling.
PeriodicCallback binds the loop late in start, so you can instantiate it wherever you want then start it later:
https://github.com/tornadoweb/tornado/blob/master/tornado/ioloop.py#L898
Got it! We'll just need to shift the `start` calls into a `call_later` then.

Resolving patch:
diff --git a/tornado/__init__.py b/tornado/__init__.py
--- a/tornado/__init__.py
+++ b/tornado/__init__.py
@@ -22,5 +22,5 @@
 # is zero for an official release, positive for a development branch,
 # or negative for a release candidate or beta (after the base version
 # number has been incremented)
-version = "6.2b2"
-version_info = (6, 2, 0, -98)
+version = "6.2"
+version_info = (6, 2, 0, 0)


Test patch:
Repository: tornadoweb/tornado

Issue:
Demos use not-yet-released features
I am trying to run the **blog** example from this repository and am getting the following error:

```
Traceback (most recent call last):

  File "./blog.py", line 53, in <module>

    class Application(tornado.web.Application):

AttributeError: module 'tornado' has no attribute 'web'
```

Reviewing other reported issues, the only reason for this is that the application was written for a previous version of Tornado (issue #3233 for example). I have made no changes to the original code. I am trying to learn this framework and am not sure how to fix this error on my own. 

I've tried running it using Python 3.11.2 and Python 3.6.15. 


Discussion:
The examples aren't out of date, they're from the future :)  They're using new features that will be released in Tornado 6.3. 

For now you can get the older versions of the demos from https://github.com/tornadoweb/tornado/tree/stable/demos
You just dismissed me by closing this issue.
You should put in the code that the features haven't been implemented. It would save us the time we spend trying to figure out why something isn't working.
BTW, the code that I was referring to came from the link you supplied which means your code isn't working
The link I supplied is a different version of the code which contains the import line that makes it work. 

I was quick to dismiss this issue because it's the third one on the same subject in a few days, but that means I should probably be leaving one of them open. I'm actively working on getting the new version of Tornado out the door, and I'll also try to update links to the demos to use the stable branch instead of master.

Resolving patch:
diff --git a/tornado/web.py b/tornado/web.py
--- a/tornado/web.py
+++ b/tornado/web.py
@@ -1904,7 +1904,7 @@ def stream_request_body(cls: Type[_RequestHandlerType]) -> Type[_RequestHandlerT
     * The regular HTTP method (``post``, ``put``, etc) will be called after
       the entire body has been read.
 
-    See the `file receiver demo <https://github.com/tornadoweb/tornado/tree/master/demos/file_upload/>`_
+    See the `file receiver demo <https://github.com/tornadoweb/tornado/tree/stable/demos/file_upload/>`_
     for example usage.
     """  # noqa: E501
     if not issubclass(cls, RequestHandler):


Test patch:
Repository: tornadoweb/tornado

Issue:
remove calls to utcnow and utcfromtimestamp
https://github.com/tornadoweb/tornado/pull/3288 fixed most of these but some remained


Discussion:
`datetime.datetime.fromtimestamp(x, datetime.timezone.utc).replace(tzinfo=None)` is a non-deprecated way to do what we were doing before, but it goes against the spirit of the deprecation. In #3288 I fixed this by using aware datetimes in most cases instead of trying to continue to use naive ones. I think we want to do the same here (and get rid of all the `replace(tzinfo=None)` calls). But I know this is untested code (or it would have been caught already), so I'm a little wary of touching it. Did you find that these `replace()` calls were necessary or were you just trying to preserve the original semantics?

Resolving patch:
diff --git a/demos/s3server/s3server.py b/demos/s3server/s3server.py
--- a/demos/s3server/s3server.py
+++ b/demos/s3server/s3server.py
@@ -138,7 +138,9 @@ def get(self):
             buckets.append(
                 {
                     "Name": name,
-                    "CreationDate": datetime.datetime.utcfromtimestamp(info.st_ctime),
+                    "CreationDate": datetime.datetime.fromtimestamp(
+                        info.st_ctime, datetime.timezone.utc
+                    ),
                 }
             )
         self.render_xml({"ListAllMyBucketsResult": {"Buckets": {"Bucket": buckets}}})
diff --git a/tornado/web.py b/tornado/web.py
--- a/tornado/web.py
+++ b/tornado/web.py
@@ -2797,7 +2797,8 @@ def set_headers(self) -> None:
         if cache_time > 0:
             self.set_header(
                 "Expires",
-                datetime.datetime.utcnow() + datetime.timedelta(seconds=cache_time),
+                datetime.datetime.now(datetime.timezone.utc)
+                + datetime.timedelta(seconds=cache_time),
             )
             self.set_header("Cache-Control", "max-age=" + str(cache_time))
 


Test patch:
diff --git a/tornado/test/web_test.py b/tornado/test/web_test.py
--- a/tornado/test/web_test.py
+++ b/tornado/test/web_test.py
@@ -1128,6 +1128,15 @@ def test_static_files(self):
         self.assertTrue(b"Disallow: /" in response.body)
         self.assertEqual(response.headers.get("Content-Type"), "text/plain")
 
+    def test_static_files_cacheable(self):
+        # Test that the version parameter triggers cache-control headers. This
+        # test is pretty weak but it gives us coverage of the code path which
+        # was important for detecting the deprecation of datetime.utcnow.
+        response = self.fetch("/robots.txt?v=12345")
+        self.assertTrue(b"Disallow: /" in response.body)
+        self.assertIn("Cache-Control", response.headers)
+        self.assertIn("Expires", response.headers)
+
     def test_static_compressed_files(self):
         response = self.fetch("/static/sample.xml.gz")
         self.assertEqual(response.headers.get("Content-Type"), "application/gzip")
Repository: open-mmlab/mmocr

Issue:
Redundant Assertion found in textsnake_targets.py
https://github.com/open-mmlab/mmocr/blob/6c1c34fb918be659dc6c61c6fd021f41b555a76f/mmocr/datasets/pipelines/textdet_targets/textsnake_targets.py#L237
https://github.com/open-mmlab/mmocr/blob/6c1c34fb918be659dc6c61c6fd021f41b555a76f/mmocr/datasets/pipelines/textdet_targets/textsnake_targets.py#L238

I think it should be
```python
assert sideline1.ndim == sideline2.ndim == 2
assert sideline1.shape[1] == sideline2.shape[1] == 2
```



Discussion:

Resolving patch:
diff --git a/mmocr/datasets/pipelines/textdet_targets/textsnake_targets.py b/mmocr/datasets/pipelines/textdet_targets/textsnake_targets.py
--- a/mmocr/datasets/pipelines/textdet_targets/textsnake_targets.py
+++ b/mmocr/datasets/pipelines/textdet_targets/textsnake_targets.py
@@ -188,6 +188,7 @@ def resample_line(self, line, n):
         assert line.shape[0] >= 2
         assert line.shape[1] == 2
         assert isinstance(n, int)
+        assert n > 0
 
         length_list = [
             norm(line[i + 1] - line[i]) for i in range(len(line) - 1)
@@ -234,8 +235,8 @@ def resample_sidelines(self, sideline1, sideline2, resample_step):
             resampled_line2 (ndarray): The resampled line 2.
         """
 
-        assert sideline1.ndim == sideline1.ndim == 2
-        assert sideline1.shape[1] == sideline1.shape[1] == 2
+        assert sideline1.ndim == sideline2.ndim == 2
+        assert sideline1.shape[1] == sideline2.shape[1] == 2
         assert sideline1.shape[0] >= 2
         assert sideline2.shape[0] >= 2
         assert isinstance(resample_step, float)
@@ -250,7 +251,7 @@ def resample_sidelines(self, sideline1, sideline2, resample_step):
         ])
 
         total_length = (length1 + length2) / 2
-        resample_point_num = int(float(total_length) / resample_step)
+        resample_point_num = max(int(float(total_length) / resample_step), 1)
 
         resampled_line1 = self.resample_line(sideline1, resample_point_num)
         resampled_line2 = self.resample_line(sideline2, resample_point_num)


Test patch:
Repository: open-mmlab/mmocr

Issue:
Make default ingnored class to -100 in SDMGRLoss
In most cases, class 0 should not be ignored.

ref https://github.com/open-mmlab/mmocr/blob/main/mmocr/models/kie/losses/sdmgr_loss.py#L17


Discussion:

Resolving patch:
diff --git a/mmocr/models/kie/losses/sdmgr_loss.py b/mmocr/models/kie/losses/sdmgr_loss.py
--- a/mmocr/models/kie/losses/sdmgr_loss.py
+++ b/mmocr/models/kie/losses/sdmgr_loss.py
@@ -14,7 +14,7 @@ class SDMGRLoss(nn.Module):
     https://arxiv.org/abs/2103.14470.
     """
 
-    def __init__(self, node_weight=1.0, edge_weight=1.0, ignore=0):
+    def __init__(self, node_weight=1.0, edge_weight=1.0, ignore=-100):
         super().__init__()
         self.loss_node = nn.CrossEntropyLoss(ignore_index=ignore)
         self.loss_edge = nn.CrossEntropyLoss(ignore_index=-1)


Test patch:
Repository: open-mmlab/mmocr

Issue:
How to train mmOCR models on total text formatted dataset?
total text is a popular scene text format dataset,you will find more info about the dataset and it's format here : https://github.com/cs-chan/Total-Text-Dataset
however in this repo for detection task you used something like coco text formatting dataset,however i want to test some of your models on total text dataset for both detection,recognition task,on the other hand we have prepared synthetic dataset which follows total text formatting,we want to use that dataset to train some of mmOCR models to solve our data problem,can we do that? our dataset is not coco text formatted and we have created it synthetically by following total text format data.i need answer to these questions : 
1. can we directly use total text format data for mmOCR detection , recognition and  keyinformation extraction task?
2.if there is no way to use total text format data directly then can you tell me how to convert total text data format in coco text format using python code?

thank you in advance <3


Discussion:

Resolving patch:
diff --git a/tools/data/textdet/totaltext_converter.py b/tools/data/textdet/totaltext_converter.py
--- a/tools/data/textdet/totaltext_converter.py
+++ b/tools/data/textdet/totaltext_converter.py
@@ -44,17 +44,17 @@ def collect_files(img_dir, gt_dir, split):
     files = []
     if split == 'training':
         for img_file in imgs_list:
-            gt_file = gt_dir + '/gt_' + osp.splitext(
-                osp.basename(img_file))[0] + '.mat'
-            # gt_file = gt_dir + '/' + osp.splitext(
-            #    osp.basename(img_file))[0] + '.png'
+            gt_file = osp.join(
+                gt_dir,
+                'poly_gt_' + osp.splitext(osp.basename(img_file))[0] + '.mat')
             files.append((img_file, gt_file))
         assert len(files), f'No images found in {img_dir}'
         print(f'Loaded {len(files)} images from {img_dir}')
     elif split == 'test':
         for img_file in imgs_list:
-            gt_file = gt_dir + '/poly_gt_' + osp.splitext(
-                osp.basename(img_file))[0] + '.mat'
+            gt_file = osp.join(
+                gt_dir,
+                'poly_gt_' + osp.splitext(osp.basename(img_file))[0] + '.mat')
             files.append((img_file, gt_file))
         assert len(files), f'No images found in {img_dir}'
         print(f'Loaded {len(files)} images from {img_dir}')
@@ -107,7 +107,7 @@ def get_contours(gt_path, split):
     words = []
     data = scio.loadmat(gt_path)
     if split == 'training':
-        data_polygt = data['gt']
+        data_polygt = data['polygt']
     elif split == 'test':
         data_polygt = data['polygt']
 


Test patch:
Repository: open-mmlab/mmocr

Issue:
LineStrParser: separator behaviour
I've a question regarding this snippet of code:
https://github.com/open-mmlab/mmocr/blob/01d8d63be945882fb2d9eaca5e1c1b39cb45f274/mmocr/datasets/utils/parser.py#L33-L36

Is there a particular reason to use these 4 lines of code instead of simply `line_str = line_str.split(self.separator)`?

I'm asking this because for my own use case I have:
- a TSV file with `filename` and `text` as keys for text recognition task
- some blank spaces in `filename` e.g. `my cropped image.png`
 
Hence, LineStrParser is configured as follows:
```python
parser=dict(
            type='LineStrParser',
            keys=['filename', 'text'],
            keys_idx=[0, 1],
            separator='\t'))
```
but with the 4-lines code snippet, the line parsing fails. Instead, with simply `line_str = line_str.split(self.separator)` everything works well.


Discussion:
Thanks for your feedback. It's maybe a bug. I will fix it soon. The original purpose of the 4-line code is that `separator` may be a `tuple` or `list`, but there is something wrong.

Resolving patch:
diff --git a/mmocr/datasets/utils/parser.py b/mmocr/datasets/utils/parser.py
--- a/mmocr/datasets/utils/parser.py
+++ b/mmocr/datasets/utils/parser.py
@@ -30,10 +30,7 @@ def __init__(self,
     def get_item(self, data_ret, index):
         map_index = index % len(data_ret)
         line_str = data_ret[map_index]
-        for split_key in self.separator:
-            if split_key != ' ':
-                line_str = line_str.replace(split_key, ' ')
-        line_str = line_str.split()
+        line_str = line_str.split(self.separator)
         if len(line_str) <= max(self.keys_idx):
             raise Exception(
                 f'key index: {max(self.keys_idx)} out of range: {line_str}')


Test patch:
Repository: open-mmlab/mmocr

Issue:
totaltext_converter.py 运行报错
你好，非常感谢团队的工作。有个问题
在提供的total_text下载链接中下载相应的图片和标注文件，但在运行过程中出现问题，

det：
Traceback (most recent call last):
  File "totaltext_converter.py", line 411, in <module>
    main()
  File "totaltext_converter.py", line 406, in main
    image_infos = collect_annotations(files, nproc=args.nproc)
  File "totaltext_converter.py", line 68, in collect_annotations
    images = mmcv.track_progress(load_img_info, files)
  File "/home/zhaoyinfa/.conda/envs/py37/lib/python3.7/site-packages/mmcv/utils/progressbar.py", line 92, in track_progress
    results.append(func(task, **kwargs))
  File "totaltext_converter.py", line 349, in load_img_info
    assert img.shape[0:2] == img_color.shape[0:2]
AssertionError

rec:
Converting training into instances_training.json
Loaded 1255 images from /home/data/datasets/OCR/rec/Totaltext/imgs/training
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 1255/1255, 64.3 task/s, elapsed: 20s, ETA:     0s
It takes 55.86578726768494s to convert totaltext annotation
Converting test into instances_test.json
Loaded 300 images from /home/data/datasets/OCR/rec/Totaltext/imgs/test
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 300/300, 54.6 task/s, elapsed: 5s, ETA:     0s
It takes 15.043917655944824s to convert totaltext annotation
Traceback (most recent call last):
  File "totaltext_converter.py", line 380, in <module>
    main()
  File "totaltext_converter.py", line 376, in main
    generate_ann(root_path, split, image_infos)
  File "totaltext_converter.py", line 300, in generate_ann
    mmcv.imwrite(dst_img, dst_img_path)
  File "/home/zhaoyinfa/.conda/envs/py37/lib/python3.7/site-packages/mmcv/image/io.py", line 258, in imwrite
    return cv2.imwrite(file_path, img, params)
cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-l1r0y34w/opencv/modules/imgcodecs/src/loadsave.cpp:803: error: (-215:Assertion failed) !_img.empty() in function 'imwrite'



Discussion:
It seems in totaltext's test set there are some invalid annotations that crop boxes outside the image. A workaround is to insert the following lines after line 296:
```
            if dst_img.shape[0] == 0 or dst_img.shape[1] == 0:
                continue
```

Resolving patch:
diff --git a/tools/data/textrecog/totaltext_converter.py b/tools/data/textrecog/totaltext_converter.py
--- a/tools/data/textrecog/totaltext_converter.py
+++ b/tools/data/textrecog/totaltext_converter.py
@@ -294,6 +294,11 @@ def generate_ann(root_path, split, image_infos):
         for anno in image_info['anno_info']:
             word = anno['word']
             dst_img = crop_img(image, anno['bbox'])
+
+            # Skip invalid annotations
+            if min(dst_img.shape) == 0:
+                continue
+
             dst_img_name = f'{src_img_root}_{index}.png'
             index += 1
             dst_img_path = osp.join(dst_image_root, dst_img_name)


Test patch: