Repository: snowflakedb/snowflake-connector-python
Issue: SNOW-739316: Snowflake Batch Results (get_result_batches())

Please answer these questions before submitting your issue. Thanks!

1. What version of Python are you using?

   python-3.8.6

2. What operating system and processor architecture are you using?

   Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.17

3. What are the component versions in the environment (`pip freeze`)?
aiofiles==0.8.0
aiohttp==3.8.3
aiosignal==1.3.1
anyio==3.6.2
APScheduler==3.8.1
asgiref==3.6.0
asn1crypto==1.5.1
async-timeout==4.0.2
asynctest==0.13.0
attrs==22.2.0
azure-core==1.26.2
azure-storage-blob==12.9.0
backports.zoneinfo==0.2.1
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
confluent-kafka==1.8.2
coverage==7.0.5
cryptography==38.0.4
defusedxml==0.7.1
ecs-logging==1.1.0
elastic-apm==6.7.2
elementpath==2.5.3
et-xmlfile==1.1.0
exceptiongroup==1.1.0
fastapi==0.75.2
fastapi-utils==0.2.1
filelock==3.9.0
frozenlist==1.3.3
greenlet==2.0.1
gunicorn==20.1.0
h11==0.14.0
idna==3.4
iniconfig==2.0.0
isodate==0.6.1
Jinja2==3.1.2
lxml==4.7.1
MarkupSafe==2.1.1
msrest==0.7.1
multidict==6.0.4
numpy==1.24.1
oauthlib==3.2.2
openpyxl==3.0.10
oscrypto==1.3.0
packaging==23.0
pandas==1.5.2
pluggy==1.0.0
prometheus-client==0.13.1
prometheus-fastapi-instrumentator==5.7.1
psutil==5.9.4
pyarrow==8.0.0
pycparser==2.21
pycryptodomex==3.16.0
pydantic==1.8.2
PyJWT==2.6.0
pyOpenSSL==22.1.0
pytest==7.2.1
pytest-asyncio==0.20.3
pytest-cov==4.0.0
pytest-env==0.8.1
pytest-mock==3.10.0
python-dateutil==2.8.2
pytz==2021.3
pytz-deprecation-shim==0.1.0.post0
PyYAML==5.4.1
requests==2.28.2
requests-oauthlib==1.3.1
schemachange==3.4.2
six==1.16.0
sly==0.5
sniffio==1.3.0
snowflake-connector-python==2.9.0
snowflake-sqlalchemy==1.4.4
SQLAlchemy==1.4.46
starlette==0.17.1
testfixtures==6.18.5
tomli==2.0.1
treebuilder==0.13
typing_extensions==4.4.0
tzdata==2022.7
tzlocal==4.2
urllib3==1.26.14
uvicorn==0.16.0
yarl==1.8.2

4. What did you do?

    It's very random in nature difficult to reproduce. General observation is 1 chunk sometime might cause an issue.
    255005: 255005: Failed to read next arrow batch: b'Expected to be able to read 149072 bytes for message body, got 81028'
  
6. What did you expect to see?

   It should fetch the batch and start processing the batch data, instead it not able to fetch batch using snowflake connector library

7. Can you set logging to DEBUG and collect the logs?

  255005: 255005: Failed to read next arrow batch: b'Expected to be able to read 149072 bytes for message body, got 81028'
  File "/app/source/sql_to_xml_convertor.py", line 114, in sql_result_iterator
    df = self.get_concatenated_dataframe(sub_array)
  File "/app/source/sql_to_xml_convertor.py", line 63, in get_concatenated_dataframe
    pyarrow_list = [arr.to_arrow() for arr in sub_array if arr.rowcount > 0]
  File "/app/source/sql_to_xml_convertor.py", line 63, in <listcomp>
    pyarrow_list = [arr.to_arrow() for arr in sub_array if arr.rowcount > 0]
  File "/opt/venv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 660, in to_arrow
    val = next(self._get_arrow_iter(connection=connection), None)
  File "/opt/venv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 646, in _get_arrow_iter
    return self._create_iter(iter_unit=IterUnit.TABLE_UNIT, connection=connection)
  File "/opt/venv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 637, in _create_iter
    loaded_data = self._load(response, iter_unit)
  File "/opt/venv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 562, in _load
    iter = PyArrowIterator(
  File "src/snowflake/connector/arrow_iterator.pyx", line 138, in snowflake.connector.arrow_iterator.PyArrowIterator.__cinit__
  File "/opt/venv/lib/python3.8/site-packages/snowflake/connector/errors.py", line 282, in errorhandler_wrapper
    raise Error.errorhandler_make_exception(
 

<!--
If you need urgent assistance reach out to support for escalated issue processing https://community.snowflake.com/s/article/How-To-Submit-a-Support-Case-in-Snowflake-Lodge
-->


Changed files:
src/snowflake/connector/vendored/urllib3/response.py

diff --git a/src/snowflake/connector/vendored/urllib3/response.py b/src/snowflake/connector/vendored/urllib3/response.py
index 0bd13d40..cb7a0646 100644
--- a/src/snowflake/connector/vendored/urllib3/response.py
+++ b/src/snowflake/connector/vendored/urllib3/response.py
@@ -213,7 +213,7 @@ class HTTPResponse(io.IOBase):
         connection=None,
         msg=None,
         retries=None,
-        enforce_content_length=False,
+        enforce_content_length=True,
         request_method=None,
         request_url=None,
         auto_close=True,