internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: microsoft/playwright-python

Issue:
Internal docs are published on the gh pages instead of the official API
We should remove everything that is not relevant to the end user!

Docs: https://microsoft.github.io/playwright-python/sync_api.html


Discussion:

Resolving patch:
diff --git a/playwright/__init__.py b/playwright/__init__.py
--- a/playwright/__init__.py
+++ b/playwright/__init__.py
@@ -11,6 +11,13 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""
+Python package `playwright` is a Python library to automate Chromium,
+Firefox and WebKit with a single API. Playwright is built to enable cross-browser
+web automation that is ever-green, capable, reliable and fast.
+For more information you'll find the documentation for the sync API [here](sync_api.html)
+and for the async API [here](async_api.html).
+"""
 
 import playwright.helper as helper
 from playwright.main import AsyncPlaywrightContextManager, SyncPlaywrightContextManager
@@ -33,3 +40,37 @@ def sync_playwright() -> SyncPlaywrightContextManager:
     "Error",
     "TimeoutError",
 ]
+
+__pdoc__ = {
+    "accessibility": False,
+    "async_base": False,
+    "browser": False,
+    "browser_context": False,
+    "browser_type": False,
+    "cdp_session": False,
+    "chromium_browser_context": False,
+    "connection": False,
+    "console_message": False,
+    "dialog": False,
+    "download": False,
+    "element_handle": False,
+    "event_context_manager": False,
+    "file_chooser": False,
+    "frame": False,
+    "helper": False,
+    "impl_to_api_mapping": False,
+    "input": False,
+    "js_handle": False,
+    "main": False,
+    "network": False,
+    "object_factory": False,
+    "page": False,
+    "path_utils": False,
+    "playwright": False,
+    "selectors": False,
+    "sync_base": False,
+    "transport": False,
+    "wait_helper": False,
+    "async_playwright": False,
+    "sync_playwright": False,
+}


Test patch:
Repository: microsoft/playwright-python

Issue:
Remove api.json and use it from the driver as subprocess call



Discussion:

Resolving patch:
diff --git a/build_package.py b/build_package.py
--- a/build_package.py
+++ b/build_package.py
@@ -99,5 +99,3 @@ def driver_version(platform: str) -> str:
             zip.write(from_location, to_location)
 
 os.remove(base_wheel_location)
-
-subprocess.check_call("python -m playwright print-api-json > api.json", shell=True)
diff --git a/scripts/documentation_provider.py b/scripts/documentation_provider.py
--- a/scripts/documentation_provider.py
+++ b/scripts/documentation_provider.py
@@ -14,6 +14,7 @@
 
 import json
 import re
+import subprocess
 from sys import stderr
 from typing import (  # type: ignore
     Any,
@@ -49,8 +50,12 @@ class DocumentationProvider:
     def __init__(self) -> None:
         self.api: Any = {}
         self.printed_entries: List[str] = []
-        with open("api.json") as json_file:
-            self.api = json.load(json_file)
+        process_output = subprocess.run(
+            ["python", "-m", "playwright", "print-api-json"],
+            check=True,
+            capture_output=True,
+        )
+        self.api = json.loads(process_output.stdout)
         self.errors: Set[str] = set()
 
     method_name_rewrites: Dict[str, str] = {


Test patch:
Repository: microsoft/playwright-python

Issue:
Error running build_package.py
When running build_package, it fails as curl is not available:
```txt
Traceback (most recent call last):
  File "build_package.py", line 43, in <module>
    subprocess.check_call(["curl", url, "-o", "driver/" + zip_file])
  File "/usr/lib/python3.8/subprocess.py", line 359, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python3.8/subprocess.py", line 340, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'curl'
```


Discussion:

Resolving patch:
diff --git a/build_package.py b/build_package.py
--- a/build_package.py
+++ b/build_package.py
@@ -18,6 +18,7 @@
 import stat
 import subprocess
 import sys
+import urllib.request
 import zipfile
 
 from playwright.path_utils import get_file_dirname
@@ -40,7 +41,7 @@ def driver_version(platform: str) -> str:
     if not os.path.exists("driver/" + zip_file):
         url = "https://playwright.azureedge.net/builds/cli/next/" + zip_file
         print("Fetching ", url)
-        subprocess.check_call(["curl", url, "-o", "driver/" + zip_file])
+        urllib.request.urlretrieve(url, "driver/" + zip_file)
 
 _dirname = get_file_dirname()
 _build_dir = _dirname / "build"


Test patch:
Repository: microsoft/playwright-python

Issue:
Interactive mode (REPL) Error !!!
**pip install playwright==0.162.2**

from playwright import sync_playwright
**playwright = sync_playwright().start()**

Traceback (most recent call last):
File "<pyshell#1>", line 1, in
playwright = sync_playwright().start()
File "C:\Python37\lib\site-packages\playwright_init_.py", line 34, in sync_playwright
return SyncPlaywrightContextManager()
File "C:\Python37\lib\site-packages\playwright\main.py", line 81, in init
self._connection = run_driver()
File "C:\Python37\lib\site-packages\playwright\main.py", line 76, in run_driver
return loop.run_until_complete(run_driver_async())
File "C:\Python37\lib\asyncio\base_events.py", line 587, in run_until_complete
return future.result()
File "C:\Python37\lib\site-packages\playwright\main.py", line 61, in run_driver_async
stderr=_get_stderr_fileno(),
File "C:\Python37\lib\site-packages\playwright\main.py", line 54, in _get_stderr_fileno
return sys.stderr.fileno()
**AttributeError: 'NoneType' object has no attribute 'fileno'**


Discussion:
This problem was raised on November 20 and updated to 0.16.2, but it has not been solved.
Relates #399

Resolving patch:
diff --git a/playwright/_impl/_transport.py b/playwright/_impl/_transport.py
--- a/playwright/_impl/_transport.py
+++ b/playwright/_impl/_transport.py
@@ -13,11 +13,25 @@
 # limitations under the License.
 
 import asyncio
+import io
 import json
 import os
 import sys
 from pathlib import Path
-from typing import Dict
+from typing import Dict, Optional
+
+
+# Sourced from: https://github.com/pytest-dev/pytest/blob/da01ee0a4bb0af780167ecd228ab3ad249511302/src/_pytest/faulthandler.py#L69-L77
+def _get_stderr_fileno() -> Optional[int]:
+    try:
+        return sys.stderr.fileno()
+    except (AttributeError, io.UnsupportedOperation):
+        # pytest-xdist monkeypatches sys.stderr with an object that is not an actual file.
+        # https://docs.python.org/3/library/faulthandler.html#issue-with-file-descriptors
+        # This is potentially dangerous, but the best we can do.
+        if not hasattr(sys, "__stderr__"):
+            return None
+        return sys.__stderr__.fileno()
 
 
 class Transport:
@@ -41,7 +55,7 @@ async def run(self) -> None:
             "run-driver",
             stdin=asyncio.subprocess.PIPE,
             stdout=asyncio.subprocess.PIPE,
-            stderr=sys.stderr,
+            stderr=_get_stderr_fileno(),
             limit=32768,
         )
         assert proc.stdout


Test patch:
Repository: microsoft/playwright-python

Issue:
Interactive mode (REPL)  (v 1.8.0a1) Error
**from playwright.sync_api import sync_playwright
playwright = sync_playwright().start()**

Traceback (most recent call last):
  File "C:\Python37\lib\site-packages\playwright\_impl\_transport.py", line 27, in _get_stderr_fileno
    return sys.stderr.fileno()
io.UnsupportedOperation: fileno

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    playwright = sync_playwright().start()
  File "C:\Python37\lib\site-packages\playwright\sync_api\_context_manager.py", line 73, in start
    return self.__enter__()
  File "C:\Python37\lib\site-packages\playwright\sync_api\_context_manager.py", line 67, in __enter__
    dispatcher_fiber.switch()
  File "C:\Python37\lib\site-packages\playwright\sync_api\_context_manager.py", line 48, in greenlet_main
    loop.run_until_complete(self._connection.run_as_sync())
  File "C:\Python37\lib\asyncio\base_events.py", line 587, in run_until_complete
    return future.result()
  File "C:\Python37\lib\site-packages\playwright\_impl\_connection.py", line 134, in run_as_sync
    await self.run()
  File "C:\Python37\lib\site-packages\playwright\_impl\_connection.py", line 139, in run
    await self._transport.run()
  File "C:\Python37\lib\site-packages\playwright\_impl\_transport.py", line 62, in run
    stderr=_get_stderr_fileno(),
  File "C:\Python37\lib\site-packages\playwright\_impl\_transport.py", line 34, in _get_stderr_fileno
    return sys.__stderr__.fileno()
AttributeError: 'NoneType' object has no attribute 'fileno'


Discussion:
python:
Python 3.7.9 (tags/v3.7.9:13c94747c7, Aug 17 2020, 18:58:18) [MSC v.1900 64 bit (AMD64)] on win32
I created a corresponding patch for it to fix it. Would be great if you could modify your `C:\Python37\lib\site-packages\playwright_impl_transport.py` locally and verify if the change in #472 would work for you.
Try to install that fix using pip :
Just run this command and try again in shell and verify.
```bash
pip install git+https://github.com/mxschmitt/playwright-python@bugfix/stderr-forwarding
``` 

@zhonghai9967 
Also in which python repl you are using it ?

Resolving patch:
diff --git a/playwright/_impl/_transport.py b/playwright/_impl/_transport.py
--- a/playwright/_impl/_transport.py
+++ b/playwright/_impl/_transport.py
@@ -29,7 +29,7 @@ def _get_stderr_fileno() -> Optional[int]:
         # pytest-xdist monkeypatches sys.stderr with an object that is not an actual file.
         # https://docs.python.org/3/library/faulthandler.html#issue-with-file-descriptors
         # This is potentially dangerous, but the best we can do.
-        if not hasattr(sys, "__stderr__"):
+        if not hasattr(sys, "__stderr__") or not sys.__stderr__:
             return None
         return sys.__stderr__.fileno()
 


Test patch:
Repository: microsoft/playwright-python

Issue:
Installation issues when using pdm
[pdm](https://github.com/frostming/pdm/) is a new tool for Python environment managing.
It works very well, but when I try to install playwright, it fails to install. You can find the details here: 
https://github.com/frostming/pdm/issues/269

The PDM developer points out that playwright seems to be missing some required fields in the wheels configs (see last comment in the linked issue).

I would highly appreciate if you could look into this issue. Cheers! 🍺


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -21,10 +21,12 @@
 from pathlib import Path
 
 import setuptools
+from auditwheel.wheeltools import InWheel
 from wheel.bdist_wheel import bdist_wheel as BDistWheelCommand
 
 driver_version = "1.9.0-1614037901000"
 
+
 def extractall(zip: zipfile.ZipFile, path: str) -> None:
     for name in zip.namelist():
         member = zip.getinfo(name)
@@ -95,6 +97,18 @@ def run(self) -> None:
                     zip.writestr("playwright/driver/README.md", "Universal Mac package")
 
         os.remove(base_wheel_location)
+        for whlfile in glob.glob("dist/*.whl"):
+
+            os.makedirs("wheelhouse", exist_ok=True)
+            with InWheel(
+                in_wheel=whlfile,
+                out_wheel=os.path.join("wheelhouse", os.path.basename(whlfile)),
+                ret_self=True,
+            ):
+                print("Updating RECORD file of %s" % whlfile)
+        shutil.rmtree("dist")
+        print("Copying new wheels")
+        shutil.move("wheelhouse", "dist")
 
 
 setuptools.setup(


Test patch:
Repository: microsoft/playwright-python

Issue:
Port the postData fix
https://github.com/microsoft/playwright/pull/5736


Discussion:

Resolving patch:
diff --git a/playwright/_impl/_network.py b/playwright/_impl/_network.py
--- a/playwright/_impl/_network.py
+++ b/playwright/_impl/_network.py
@@ -85,11 +85,12 @@ def post_data_json(self) -> Optional[Any]:
         if not post_data:
             return None
         content_type = self.headers["content-type"]
-        if not content_type:
-            return None
         if content_type == "application/x-www-form-urlencoded":
             return dict(parse.parse_qsl(post_data))
-        return json.loads(post_data)
+        try:
+            return json.loads(post_data)
+        except Exception:
+            raise Error(f"POST data is not a valid JSON object: {post_data}")
 
     @property
     def post_data_buffer(self) -> Optional[bytes]:


Test patch:
diff --git a/tests/async/test_network.py b/tests/async/test_network.py
--- a/tests/async/test_network.py
+++ b/tests/async/test_network.py
@@ -279,6 +279,44 @@ async def test_should_be_undefined_when_there_is_no_post_data(page, server):
     assert response.request.post_data_json is None
 
 
+async def test_should_return_post_data_without_content_type(page, server):
+    await page.goto(server.EMPTY_PAGE)
+    async with page.expect_request("**/*") as request_info:
+        await page.evaluate(
+            """({url}) => {
+            const request = new Request(url, {
+                method: 'POST',
+                body: JSON.stringify({ value: 42 }),
+            });
+            request.headers.set('content-type', '');
+            return fetch(request);
+        }""",
+            {"url": server.PREFIX + "/title.html"},
+        )
+    request = await request_info.value
+    assert request.post_data_json == {"value": 42}
+
+
+async def test_should_throw_on_invalid_json_in_post_data(page, server):
+    await page.goto(server.EMPTY_PAGE)
+    async with page.expect_request("**/*") as request_info:
+        await page.evaluate(
+            """({url}) => {
+            const request = new Request(url, {
+                method: 'POST',
+                body: '<not a json>',
+            });
+            request.headers.set('content-type', '');
+            return fetch(request);
+        }""",
+            {"url": server.PREFIX + "/title.html"},
+        )
+    request = await request_info.value
+    with pytest.raises(Error) as exc_info:
+        print(request.post_data_json)
+    assert "POST data is not a valid JSON object: <not a json>" in str(exc_info.value)
+
+
 async def test_should_work_with_binary_post_data(page, server):
     await page.goto(server.EMPTY_PAGE)
     server.set_route("/post", lambda req: req.finish())
Repository: microsoft/playwright-python

Issue:
[BUG] misleading error message "browser was not found" if browser fails to start for other reasons
# Environment
playwright==1.9.2
ubuntu 20.04 in docker
chromium v854489 installed via `python3 -m playwright install`

# Problem description:
Unintentionally, the test attempted to launch headful chromium, launch failed with misleading error message:
```
"chromium" browser was not found.
Please complete Playwright installation via running
    "python -m playwright install"
```
Launching browser as `chromium.launch(headless=True)` was succesful, so the problem clearly was not with browser installation, but browser launch inself (most probably, unavailable X11)

# Expected behavior:
browser stderr captured and displayed



Discussion:

Resolving patch:
diff --git a/playwright/_impl/_browser_type.py b/playwright/_impl/_browser_type.py
--- a/playwright/_impl/_browser_type.py
+++ b/playwright/_impl/_browser_type.py
@@ -69,7 +69,7 @@ async def launch(
         try:
             return from_channel(await self._channel.send("launch", params))
         except Exception as e:
-            if f"{self.name}-" in str(e):
+            if "because executable doesn't exist" in str(e):
                 raise not_installed_error(f'"{self.name}" browser was not found.')
             raise e
 
@@ -124,7 +124,7 @@ async def launch_persistent_context(
             context._options = params
             return context
         except Exception as e:
-            if f"{self.name}-" in str(e):
+            if "because executable doesn't exist" in str(e):
                 raise not_installed_error(f'"{self.name}" browser was not found.')
             raise e
 
diff --git a/playwright/_impl/_helper.py b/playwright/_impl/_helper.py
--- a/playwright/_impl/_helper.py
+++ b/playwright/_impl/_helper.py
@@ -203,7 +203,7 @@ def is_safe_close_error(error: Exception) -> bool:
 
 
 def not_installed_error(message: str) -> Exception:
-    return Exception(
+    return Error(
         f"""
 ================================================================================
 {message}


Test patch:
diff --git a/tests/async/test_launcher.py b/tests/async/test_launcher.py
--- a/tests/async/test_launcher.py
+++ b/tests/async/test_launcher.py
@@ -121,3 +121,12 @@ async def test_browser_close_should_be_callable_twice(browser_type, launch_argum
         browser.close(),
     )
     await browser.close()
+
+
+async def test_browser_launch_non_existing_executable_path_shows_install_msg(
+    browser_type,
+    tmpdir,
+):
+    with pytest.raises(Error) as exc_info:
+        await browser_type.launch(executable_path=tmpdir.join("executable"))
+    assert "python -m playwright install" in exc_info.value.message
Repository: microsoft/playwright-python

Issue:
Convert None to null when serializing expression arguments
I've spent a good two full days trying to figure why it was not working. 

We have a custom function attached to the window object that I was trying invoke with a deep dictionary object. While debugging I realized that when the framework serializes expression arguments it converts `None` to `undefined` instead of `null`.

In our case we explicitly check for `null` (inside the browser) and that was the reason it was failing.

I understand that you have both options (`undefined` and `null`) in javascript world while there is only `None` in python world. So I guess you had to choose one or another. My question is if it was the right choice here. 

Also, it is a very subtle issue and it is not documented anywhere. 

Below is the code in question.

https://github.com/microsoft/playwright-python/blob/6854c40ad4bdc8e1fbc8016998fdcf179d7120aa/playwright/_impl/_js_handle.py#L101


Discussion:

Resolving patch:
diff --git a/playwright/_impl/_js_handle.py b/playwright/_impl/_js_handle.py
--- a/playwright/_impl/_js_handle.py
+++ b/playwright/_impl/_js_handle.py
@@ -98,7 +98,7 @@ def serialize_value(value: Any, handles: List[JSHandle], depth: int) -> Any:
     if depth > 100:
         raise Error("Maximum argument depth exceeded")
     if value is None:
-        return dict(v="undefined")
+        return dict(v="null")
     if isinstance(value, float):
         if value == float("inf"):
             return dict(v="Infinity")


Test patch:
diff --git a/tests/async/test_evaluate.py b/tests/async/test_evaluate.py
--- a/tests/async/test_evaluate.py
+++ b/tests/async/test_evaluate.py
@@ -142,7 +142,7 @@ async def test_evaluate_return_complex_objects(page):
 
 async def test_evaluate_accept_none_as_one_of_multiple_parameters(page):
     result = await page.evaluate(
-        '({ a, b }) => Object.is(a, undefined) && Object.is(b, "foo")',
+        '({ a, b }) => Object.is(a, null) && Object.is(b, "foo")',
         {"a": None, "b": "foo"},
     )
     assert result
Repository: microsoft/playwright-python

Issue:
[Bug]: `record_har_omit_content` does not work properly
### Playwright version

1.15.3

### Operating system

Linux

### What browsers are you seeing the problem on?

Chromium, Firefox, WebKit

### Other information

Repo to present bug:
https://github.com/qwark97/har_omit_content_bug

Bug occurs also outside the docker image

### What happened? / Describe the bug

Using `record_har_omit_content` as a `new_page` parameter can manipulate presence of the `text` filed inside `entry.response.content` object in `.har` file.

Not using this parameter (defaults to `False` https://playwright.dev/python/docs/api/class-browser#browser-new-page) allows to see `text` inside `.har` file. Using `record_har_omit_content=True` also works as expected - `text` is absent. Unfortunatelly, passing `record_har_omit_content=False` explicitely **does not** work as expected -> `.har` file **will not** contain `text` filed.

It also looks like passing anything except explicit `None` as a `record_har_omit_content` value (type doesn't matter) will cause with missing `text` filed

### Code snippet to reproduce your bug

_No response_

### Relevant log output

_No response_


Discussion:

Resolving patch:
diff --git a/playwright/_impl/_browser.py b/playwright/_impl/_browser.py
--- a/playwright/_impl/_browser.py
+++ b/playwright/_impl/_browser.py
@@ -210,7 +210,7 @@ async def normalize_context_params(is_sync: bool, params: Dict) -> None:
     if "recordHarPath" in params:
         params["recordHar"] = {"path": str(params["recordHarPath"])}
         if "recordHarOmitContent" in params:
-            params["recordHar"]["omitContent"] = True
+            params["recordHar"]["omitContent"] = params["recordHarOmitContent"]
             del params["recordHarOmitContent"]
         del params["recordHarPath"]
     if "recordVideoDir" in params:


Test patch:
diff --git a/tests/async/test_har.py b/tests/async/test_har.py
--- a/tests/async/test_har.py
+++ b/tests/async/test_har.py
@@ -40,11 +40,24 @@ async def test_should_omit_content(browser, server, tmpdir):
         data = json.load(f)
         assert "log" in data
         log = data["log"]
-
         content1 = log["entries"][0]["response"]["content"]
         assert "text" not in content1
 
 
+async def test_should_not_omit_content(browser, server, tmpdir):
+    path = os.path.join(tmpdir, "log.har")
+    context = await browser.new_context(
+        record_har_path=path, record_har_omit_content=False
+    )
+    page = await context.new_page()
+    await page.goto(server.PREFIX + "/har.html")
+    await context.close()
+    with open(path) as f:
+        data = json.load(f)
+        content1 = data["log"]["entries"][0]["response"]["content"]
+        assert "text" in content1
+
+
 async def test_should_include_content(browser, server, tmpdir):
     path = os.path.join(tmpdir, "log.har")
     context = await browser.new_context(record_har_path=path)
Repository: microsoft/playwright-python

Issue:
Fixing a pyee DeprecationWarning
pyee.AsyncIOEventEmitter was moved to pyee.asyncio.AsyncIOEventEmitter, so this PR just fixes two imports


Discussion:
[![CLA assistant check](https://cla.opensource.microsoft.com/pull/badge/signed)](https://cla.opensource.microsoft.com/microsoft/playwright-python?pullRequest=1117) <br/>All CLA requirements met.
I think we should also bump pyee to `>=9.0.0` inside `setup.py` and `meta.yaml` otherwise it would break.
Went ahead and updated those files
It seems like version 9.0 is not available on conda-forge yet: https://github.com/conda-forge/pyee-feedstock/ So we need to look into this one as well. I can try to dig into it tomorrow.
I opened https://github.com/conda-forge/pyee-feedstock/pull/15 to fix the pyee conda-forge package. Since this is only about a deprecation warning, we can wait for that a few days.
Thanks! And, definitely no urgency needed. Just wanted to try and get this warning fixed since it comes up every time I run my test scripts

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -208,9 +208,9 @@ def _download_and_extract_local_driver(
     packages=find_packages(exclude=["tests*"]),
     include_package_data=True,
     install_requires=[
-        "websockets>=8.1",
-        "greenlet>=1.0.0",
-        "pyee>=8.0.1",
+        "websockets==10.1",
+        "greenlet==1.1.2",
+        "pyee==8.1.0",
         "typing-extensions;python_version<='3.8'",
     ],
     classifiers=[


Test patch: