Repository: googleapis/python-bigquery (python)

Issue:
pyarrow import error in 3.20.0
#### Environment details

  - OS type and version: MacOS 14.4.1 (23E224)
  - Python version: 3.10
  - pip version: 24.0
  - `google-cloud-bigquery` version: 3.20.0 

#### Steps to reproduce
Create a new virtualenv, install google-cloud-bigquery==3.20.0, attempt to `import google.cloud.bigquery`

```
$ virtualenv -p python3.10 venv
created virtual environment CPython3.10.14.final.0-64 in 322ms
  creator CPython3macOsBrew(dest=/Users/me/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/me/Library/Application Support/virtualenv)
    added seed packages: pip==24.0, setuptools==69.1.1, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
```  
 
```
$ source venv/bin/activate
```
```
$ pip install -i https://pypi.python.org/simple google-cloud-bigquery==3.20.0
Looking in indexes: https://pypi.python.org/simple
Collecting google-cloud-bigquery==3.20.0
  Using cached google_cloud_bigquery-3.20.0-py2.py3-none-any.whl.metadata (8.9 kB)
Collecting google-api-core!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1 (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-bigquery==3.20.0)
  Using cached google_api_core-2.18.0-py3-none-any.whl.metadata (2.7 kB)
Collecting google-auth<3.0.0dev,>=2.14.1 (from google-cloud-bigquery==3.20.0)
  Using cached google_auth-2.29.0-py2.py3-none-any.whl.metadata (4.7 kB)
Collecting google-cloud-core<3.0.0dev,>=1.6.0 (from google-cloud-bigquery==3.20.0)
  Using cached google_cloud_core-2.4.1-py2.py3-none-any.whl.metadata (2.7 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery==3.20.0)
  Using cached google_resumable_media-2.7.0-py2.py3-none-any.whl.metadata (2.2 kB)
Collecting packaging>=20.0.0 (from google-cloud-bigquery==3.20.0)
  Using cached packaging-24.0-py3-none-any.whl.metadata (3.2 kB)
Collecting python-dateutil<3.0dev,>=2.7.2 (from google-cloud-bigquery==3.20.0)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting requests<3.0.0dev,>=2.21.0 (from google-cloud-bigquery==3.20.0)
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-bigquery==3.20.0)
  Using cached googleapis_common_protos-1.63.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0.dev0,>=3.19.5 (from google-api-core!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-bigquery==3.20.0)
  Using cached protobuf-4.25.3-cp37-abi3-macosx_10_9_universal2.whl.metadata (541 bytes)
Collecting proto-plus<2.0.0dev,>=1.22.3 (from google-api-core!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-bigquery==3.20.0)
  Using cached proto_plus-1.23.0-py3-none-any.whl.metadata (2.2 kB)
Collecting grpcio<2.0dev,>=1.33.2 (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-bigquery==3.20.0)
  Using cached grpcio-1.62.1-cp310-cp310-macosx_12_0_universal2.whl.metadata (4.0 kB)
Collecting grpcio-status<2.0.dev0,>=1.33.2 (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-bigquery==3.20.0)
  Using cached grpcio_status-1.62.1-py3-none-any.whl.metadata (1.3 kB)
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3.0.0dev,>=2.14.1->google-cloud-bigquery==3.20.0)
  Using cached cachetools-5.3.3-py3-none-any.whl.metadata (5.3 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3.0.0dev,>=2.14.1->google-cloud-bigquery==3.20.0)
  Using cached pyasn1_modules-0.4.0-py3-none-any.whl.metadata (3.4 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3.0.0dev,>=2.14.1->google-cloud-bigquery==3.20.0)
  Using cached rsa-4.9-py3-none-any.whl.metadata (4.2 kB)
Collecting google-crc32c<2.0dev,>=1.0 (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery==3.20.0)
  Using cached google_crc32c-1.5.0-cp310-cp310-macosx_10_9_universal2.whl.metadata (2.3 kB)
Collecting six>=1.5 (from python-dateutil<3.0dev,>=2.7.2->google-cloud-bigquery==3.20.0)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting charset-normalizer<4,>=2 (from requests<3.0.0dev,>=2.21.0->google-cloud-bigquery==3.20.0)
  Using cached charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3.0.0dev,>=2.21.0->google-cloud-bigquery==3.20.0)
  Using cached idna-3.6-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3.0.0dev,>=2.21.0->google-cloud-bigquery==3.20.0)
  Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests<3.0.0dev,>=2.21.0->google-cloud-bigquery==3.20.0)
  Using cached certifi-2024.2.2-py3-none-any.whl.metadata (2.2 kB)
Collecting pyasn1<0.7.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3.0.0dev,>=2.14.1->google-cloud-bigquery==3.20.0)
  Using cached pyasn1-0.6.0-py2.py3-none-any.whl.metadata (8.3 kB)
Using cached google_cloud_bigquery-3.20.0-py2.py3-none-any.whl (233 kB)
Using cached google_api_core-2.18.0-py3-none-any.whl (138 kB)
Using cached google_auth-2.29.0-py2.py3-none-any.whl (189 kB)
Using cached google_cloud_core-2.4.1-py2.py3-none-any.whl (29 kB)
Using cached google_resumable_media-2.7.0-py2.py3-none-any.whl (80 kB)
Using cached packaging-24.0-py3-none-any.whl (53 kB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached cachetools-5.3.3-py3-none-any.whl (9.3 kB)
Using cached certifi-2024.2.2-py3-none-any.whl (163 kB)
Using cached charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl (120 kB)
Using cached google_crc32c-1.5.0-cp310-cp310-macosx_10_9_universal2.whl (32 kB)
Using cached googleapis_common_protos-1.63.0-py2.py3-none-any.whl (229 kB)
Using cached grpcio-1.62.1-cp310-cp310-macosx_12_0_universal2.whl (10.0 MB)
Using cached grpcio_status-1.62.1-py3-none-any.whl (14 kB)
Using cached idna-3.6-py3-none-any.whl (61 kB)
Using cached proto_plus-1.23.0-py3-none-any.whl (48 kB)
Using cached protobuf-4.25.3-cp37-abi3-macosx_10_9_universal2.whl (394 kB)
Using cached pyasn1_modules-0.4.0-py3-none-any.whl (181 kB)
Using cached rsa-4.9-py3-none-any.whl (34 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Using cached urllib3-2.2.1-py3-none-any.whl (121 kB)
Using cached pyasn1-0.6.0-py2.py3-none-any.whl (85 kB)
Installing collected packages: urllib3, six, pyasn1, protobuf, packaging, idna, grpcio, google-crc32c, charset-normalizer, certifi, cachetools, rsa, requests, python-dateutil, pyasn1-modules, proto-plus, googleapis-common-protos, google-resumable-media, grpcio-status, google-auth, google-api-core, google-cloud-core, google-cloud-bigquery
Successfully installed cachetools-5.3.3 certifi-2024.2.2 charset-normalizer-3.3.2 google-api-core-2.18.0 google-auth-2.29.0 google-cloud-bigquery-3.20.0 google-cloud-core-2.4.1 google-crc32c-1.5.0 google-resumable-media-2.7.0 googleapis-common-protos-1.63.0 grpcio-1.62.1 grpcio-status-1.62.1 idna-3.6 packaging-24.0 proto-plus-1.23.0 protobuf-4.25.3 pyasn1-0.6.0 pyasn1-modules-0.4.0 python-dateutil-2.9.0.post0 requests-2.31.0 rsa-4.9 six-1.16.0 urllib3-2.2.1
```

```
$ python -c "import google.cloud.bigquery"
Traceback (most recent call last):
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/_versions_helpers.py", line 75, in try_import
    import pyarrow
ModuleNotFoundError: No module named 'pyarrow'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/__init__.py", line 35, in <module>
    from google.cloud.bigquery.client import Client
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/client.py", line 69, in <module>
    from google.cloud.bigquery import _job_helpers
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/_job_helpers.py", line 47, in <module>
    from google.cloud.bigquery import job
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/job/__init__.py", line 27, in <module>
    from google.cloud.bigquery.job.copy_ import CopyJob
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/job/copy_.py", line 22, in <module>
    from google.cloud.bigquery.table import TableReference
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/table.py", line 62, in <module>
    from google.cloud.bigquery import _pandas_helpers
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 52, in <module>
    pyarrow = _versions_helpers.PYARROW_VERSIONS.try_import(raise_if_error=True)
  File "/Users/me/venv/lib/python3.10/site-packages/google/cloud/bigquery/_versions_helpers.py", line 78, in try_import
    raise exceptions.LegacyPyarrowError(
google.cloud.bigquery.exceptions.LegacyPyarrowError: pyarrow package not found. Install pyarrow version >= 3.0.0.
``` 

Pull request description:
fix: make `pyarrow` an optional dependency post-3.20.0 yanked release
Partial rollback of https://github.com/googleapis/python-bigquery/commit/0ac6e9bf186945832f5dcdf5a4d95667b4da223e

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #1877 🦕


Patch:
diff --git a/google/cloud/bigquery/_pandas_helpers.py b/google/cloud/bigquery/_pandas_helpers.py
index 9f8dcfde..3b58d373 100644
--- a/google/cloud/bigquery/_pandas_helpers.py
+++ b/google/cloud/bigquery/_pandas_helpers.py
@@ -32,7 +32,7 @@ try:
     import pandas  # type: ignore
 
     pandas_import_exception = None
-except ImportError as exc:  # pragma: NO COVER
+except ImportError as exc:
     pandas = None
     pandas_import_exception = exc
 else:
@@ -44,25 +44,21 @@ try:
     date_dtype_name = db_dtypes.DateDtype.name
     time_dtype_name = db_dtypes.TimeDtype.name
     db_dtypes_import_exception = None
-except ImportError as exc:  # pragma: NO COVER
+except ImportError as exc:
     db_dtypes = None
     db_dtypes_import_exception = exc
     date_dtype_name = time_dtype_name = ""  # Use '' rather than None because pytype
 
-pyarrow = _versions_helpers.PYARROW_VERSIONS.try_import(raise_if_error=True)
-from pyarrow import ArrowTypeError  # type: ignore # noqa: E402
-
-_BIGNUMERIC_SUPPORT = False
-if pyarrow is not None:  # pragma: NO COVER
-    _BIGNUMERIC_SUPPORT = True
+pyarrow = _versions_helpers.PYARROW_VERSIONS.try_import()
 
 try:
     # _BaseGeometry is used to detect shapely objevys in `bq_to_arrow_array`
     from shapely.geometry.base import BaseGeometry as _BaseGeometry  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     # No shapely, use NoneType for _BaseGeometry as a placeholder.
     _BaseGeometry = type(None)
 else:
+    # We don't have any unit test sessions that install shapely but not pandas.
     if pandas is not None:  # pragma: NO COVER
 
         def _to_wkb():
@@ -309,10 +305,10 @@ def bq_to_arrow_array(series, bq_field):
         if field_type_upper in schema._STRUCT_TYPES:
             return pyarrow.StructArray.from_pandas(series, type=arrow_type)
         return pyarrow.Array.from_pandas(series, type=arrow_type)
-    except ArrowTypeError:  # pragma: NO COVER
+    except pyarrow.ArrowTypeError:
         msg = f"""Error converting Pandas column with name: "{series.name}" and datatype: "{series.dtype}" to an appropriate pyarrow datatype: Array, ListArray, or StructArray"""
         _LOGGER.error(msg)
-        raise ArrowTypeError(msg)
+        raise pyarrow.ArrowTypeError(msg)
 
 
 def get_column_or_index(dataframe, name):
diff --git a/google/cloud/bigquery/_pyarrow_helpers.py b/google/cloud/bigquery/_pyarrow_helpers.py
index 06509cc9..3c745a61 100644
--- a/google/cloud/bigquery/_pyarrow_helpers.py
+++ b/google/cloud/bigquery/_pyarrow_helpers.py
@@ -20,7 +20,7 @@ from packaging import version
 
 try:
     import pyarrow  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pyarrow = None
 
 
@@ -49,7 +49,7 @@ def pyarrow_timestamp():
 _BQ_TO_ARROW_SCALARS = {}
 _ARROW_SCALAR_IDS_TO_BQ = {}
 
-if pyarrow:  # pragma: NO COVER
+if pyarrow:
     # This dictionary is duplicated in bigquery_storage/test/unite/test_reader.py
     # When modifying it be sure to update it there as well.
     # Note(todo!!): type "BIGNUMERIC"'s matching pyarrow type is added in _pandas_helpers.py
diff --git a/google/cloud/bigquery/_tqdm_helpers.py b/google/cloud/bigquery/_tqdm_helpers.py
index cb81bd8f..22ccee97 100644
--- a/google/cloud/bigquery/_tqdm_helpers.py
+++ b/google/cloud/bigquery/_tqdm_helpers.py
@@ -23,11 +23,14 @@ import warnings
 
 try:
     import tqdm  # type: ignore
-    import tqdm.notebook as notebook  # type: ignore
-
-except ImportError:  # pragma: NO COVER
+except ImportError:
     tqdm = None
 
+try:
+    import tqdm.notebook as tqdm_notebook  # type: ignore
+except ImportError:
+    tqdm_notebook = None
+
 if typing.TYPE_CHECKING:  # pragma: NO COVER
     from google.cloud.bigquery import QueryJob
     from google.cloud.bigquery.table import RowIterator
@@ -42,7 +45,7 @@ _PROGRESS_BAR_UPDATE_INTERVAL = 0.5
 
 def get_progress_bar(progress_bar_type, description, total, unit):
     """Construct a tqdm progress bar object, if tqdm is installed."""
-    if tqdm is None:
+    if tqdm is None or tqdm_notebook is None and progress_bar_type == "tqdm_notebook":
         if progress_bar_type is not None:
             warnings.warn(_NO_TQDM_ERROR, UserWarning, stacklevel=3)
         return None
@@ -58,7 +61,7 @@ def get_progress_bar(progress_bar_type, description, total, unit):
                 unit=unit,
             )
         elif progress_bar_type == "tqdm_notebook":
-            return notebook.tqdm(
+            return tqdm_notebook.tqdm(
                 bar_format="{l_bar}{bar}|",
                 desc=description,
                 file=sys.stdout,
diff --git a/google/cloud/bigquery/_versions_helpers.py b/google/cloud/bigquery/_versions_helpers.py
index 4ff4b970..50d5961b 100644
--- a/google/cloud/bigquery/_versions_helpers.py
+++ b/google/cloud/bigquery/_versions_helpers.py
@@ -73,7 +73,7 @@ class PyarrowVersions:
         """
         try:
             import pyarrow
-        except ImportError as exc:  # pragma: NO COVER
+        except ImportError as exc:
             if raise_if_error:
                 raise exceptions.LegacyPyarrowError(
                     "pyarrow package not found. Install pyarrow version >="
@@ -212,7 +212,7 @@ class PandasVersions:
         """
         try:
             import pandas
-        except ImportError as exc:  # pragma: NO COVER
+        except ImportError as exc:
             if raise_if_error:
                 raise exceptions.LegacyPandasError(
                     "pandas package not found. Install pandas version >="
diff --git a/google/cloud/bigquery/job/query.py b/google/cloud/bigquery/job/query.py
index 83d2751c..e92e9cb9 100644
--- a/google/cloud/bigquery/job/query.py
+++ b/google/cloud/bigquery/job/query.py
@@ -56,14 +56,9 @@ from google.cloud.bigquery.job.base import _JobReference
 
 try:
     import pandas  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pandas = None
 
-try:
-    import db_dtypes  # type: ignore
-except ImportError:  # pragma: NO COVER
-    db_dtypes = None
-
 if typing.TYPE_CHECKING:  # pragma: NO COVER
     # Assumption: type checks are only used by library developers and CI environments
     # that have all optional dependencies installed, thus no conditional imports.
diff --git a/google/cloud/bigquery/magics/magics.py b/google/cloud/bigquery/magics/magics.py
index 8464c879..6e6b2196 100644
--- a/google/cloud/bigquery/magics/magics.py
+++ b/google/cloud/bigquery/magics/magics.py
@@ -95,7 +95,7 @@ try:
     import IPython  # type: ignore
     from IPython import display  # type: ignore
     from IPython.core import magic_arguments  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     raise ImportError("This module can only be loaded in IPython.")
 
 from google.api_core import client_info
diff --git a/google/cloud/bigquery/table.py b/google/cloud/bigquery/table.py
index b3be4ff9..c002822f 100644
--- a/google/cloud/bigquery/table.py
+++ b/google/cloud/bigquery/table.py
@@ -26,17 +26,17 @@ import warnings
 
 try:
     import pandas  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pandas = None
 
 try:
     import pyarrow  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pyarrow = None
 
 try:
     import db_dtypes  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     db_dtypes = None
 
 try:
diff --git a/noxfile.py b/noxfile.py
index 548690af..3adb4ba7 100644
--- a/noxfile.py
+++ b/noxfile.py
@@ -86,7 +86,7 @@ def default(session, install_extras=True):
         install_target = ".[all]"
     else:
         install_target = "."
-    session.install("-e", install_target)
+    session.install("-e", install_target, "-c", constraints_path)
     session.run("python", "-m", "pip", "freeze")
 
     # Run py.test against the unit tests.
@@ -115,14 +115,15 @@ def unit(session):
 def unit_noextras(session):
     """Run the unit test suite."""
 
-    # Install optional dependencies that are out-of-date.
+    # Install optional dependencies that are out-of-date to see that
+    # we fail gracefully.
     # https://github.com/googleapis/python-bigquery/issues/933
-    # There is no pyarrow 1.0.0 package for Python 3.9.
-
+    #
+    # We only install this extra package on one of the two Python versions
+    # so that it continues to be an optional dependency.
+    # https://github.com/googleapis/python-bigquery/issues/1877
     if session.python == UNIT_TEST_PYTHON_VERSIONS[0]:
-        session.install("pyarrow>=3.0.0")
-    elif session.python == UNIT_TEST_PYTHON_VERSIONS[-1]:
-        session.install("pyarrow")
+        session.install("pyarrow==1.0.0")
 
     default(session, install_extras=False)
 
diff --git a/samples/snippets/requirements.txt b/samples/snippets/requirements.txt
index b3347499..af9436c5 100644
--- a/samples/snippets/requirements.txt
+++ b/samples/snippets/requirements.txt
@@ -1,1 +1,2 @@
-google-cloud-bigquery==3.19.0
\ No newline at end of file
+# samples/snippets should be runnable with no "extras"
+google-cloud-bigquery==3.19.0


Test patch:
diff --git a/samples/desktopapp/requirements-test.txt b/samples/desktopapp/requirements-test.txt
index 413a7fd4..9142d490 100644
--- a/samples/desktopapp/requirements-test.txt
+++ b/samples/desktopapp/requirements-test.txt
@@ -2,4 +2,3 @@ google-cloud-testutils==1.4.0
 pytest===7.4.4; python_version == '3.7'
 pytest==8.1.1; python_version >= '3.8'
 mock==5.1.0
-pyarrow>=3.0.0
\ No newline at end of file
diff --git a/samples/snippets/requirements-test.txt b/samples/snippets/requirements-test.txt
index 413a7fd4..0343ab89 100644
--- a/samples/snippets/requirements-test.txt
+++ b/samples/snippets/requirements-test.txt
@@ -1,5 +1,5 @@
+# samples/snippets should be runnable with no "extras"
 google-cloud-testutils==1.4.0
 pytest===7.4.4; python_version == '3.7'
 pytest==8.1.1; python_version >= '3.8'
 mock==5.1.0
-pyarrow>=3.0.0
\ No newline at end of file
diff --git a/testing/constraints-3.11.txt b/testing/constraints-3.11.txt
index e80ca0cc..e69de29b 100644
--- a/testing/constraints-3.11.txt
+++ b/testing/constraints-3.11.txt
@@ -1,1 +0,0 @@
-pyarrow>=3.0.0
\ No newline at end of file
diff --git a/testing/constraints-3.12.txt b/testing/constraints-3.12.txt
index e80ca0cc..e69de29b 100644
--- a/testing/constraints-3.12.txt
+++ b/testing/constraints-3.12.txt
@@ -1,1 +0,0 @@
-pyarrow>=3.0.0
\ No newline at end of file
diff --git a/testing/constraints-3.7.txt b/testing/constraints-3.7.txt
index 1fc7c683..d64e06cc 100644
--- a/testing/constraints-3.7.txt
+++ b/testing/constraints-3.7.txt
@@ -27,9 +27,9 @@ packaging==20.0.0
 pandas==1.1.0
 proto-plus==1.22.0
 protobuf==3.19.5
-pyarrow>=3.0.0
+pyarrow==3.0.0
 python-dateutil==2.7.3
 requests==2.21.0
 Shapely==1.8.4
 six==1.13.0
-tqdm==4.7.4
\ No newline at end of file
+tqdm==4.7.4
diff --git a/tests/unit/job/test_query_pandas.py b/tests/unit/job/test_query_pandas.py
index 1473ef28..3a5d92db 100644
--- a/tests/unit/job/test_query_pandas.py
+++ b/tests/unit/job/test_query_pandas.py
@@ -19,53 +19,38 @@ from unittest import mock
 
 import pytest
 
+from ..helpers import make_connection
+from .helpers import _make_client
+from .helpers import _make_job_resource
 
 try:
     from google.cloud import bigquery_storage
     import google.cloud.bigquery_storage_v1.reader
     import google.cloud.bigquery_storage_v1.services.big_query_read.client
-except (ImportError, AttributeError):  # pragma: NO COVER
+except (ImportError, AttributeError):
     bigquery_storage = None
 
-try:
-    import pandas
-except (ImportError, AttributeError):  # pragma: NO COVER
-    pandas = None
 try:
     import shapely
-except (ImportError, AttributeError):  # pragma: NO COVER
+except (ImportError, AttributeError):
     shapely = None
 try:
     import geopandas
-except (ImportError, AttributeError):  # pragma: NO COVER
+except (ImportError, AttributeError):
     geopandas = None
 try:
     import tqdm
-except (ImportError, AttributeError):  # pragma: NO COVER
+except (ImportError, AttributeError):
     tqdm = None
 
-try:
-    import importlib.metadata as metadata
-except ImportError:
-    import importlib_metadata as metadata
-
-from ..helpers import make_connection
-from .helpers import _make_client
-from .helpers import _make_job_resource
-
-if pandas is not None:
-    PANDAS_INSTALLED_VERSION = metadata.version("pandas")
-else:
-    PANDAS_INSTALLED_VERSION = "0.0.0"
-
-pandas = pytest.importorskip("pandas")
-
 try:
     import pyarrow
     import pyarrow.types
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pyarrow = None
 
+pandas = pytest.importorskip("pandas")
+
 
 @pytest.fixture
 def table_read_options_kwarg():
@@ -660,7 +645,10 @@ def test_to_dataframe_bqstorage_no_pyarrow_compression():
     )
 
 
-@pytest.mark.skipif(PANDAS_INSTALLED_VERSION[0:2] not in ["0.", "1."], reason="")
+@pytest.mark.skipif(
+    pandas.__version__.startswith("2."),
+    reason="pandas 2.0 changes some default dtypes and we haven't update the test to account for those",
+)
 @pytest.mark.skipif(pyarrow is None, reason="Requires `pyarrow`")
 def test_to_dataframe_column_dtypes():
     from google.cloud.bigquery.job import QueryJob as target_class
diff --git a/tests/unit/test__pandas_helpers.py b/tests/unit/test__pandas_helpers.py
index 24438462..5c13669f 100644
--- a/tests/unit/test__pandas_helpers.py
+++ b/tests/unit/test__pandas_helpers.py
@@ -30,12 +30,12 @@ try:
     import pandas
     import pandas.api.types
     import pandas.testing
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pandas = None
 
 try:
     import geopandas
-except ImportError:  # pragma: NO COVER
+except ImportError:
     geopandas = None
 
 import pytest
@@ -46,18 +46,19 @@ from google.cloud.bigquery import exceptions
 from google.cloud.bigquery import _pyarrow_helpers
 from google.cloud.bigquery import _versions_helpers
 from google.cloud.bigquery import schema
-from google.cloud.bigquery._pandas_helpers import _BIGNUMERIC_SUPPORT
 
 pyarrow = _versions_helpers.PYARROW_VERSIONS.try_import()
 
 if pyarrow:
     import pyarrow.parquet
     import pyarrow.types
-    from pyarrow import ArrowTypeError  # type: ignore # noqa: E402
-else:  # pragma: NO COVER
+
+    _BIGNUMERIC_SUPPORT = True
+else:
     # Mock out pyarrow when missing, because methods from pyarrow.types are
     # used in test parameterization.
     pyarrow = mock.Mock()
+    _BIGNUMERIC_SUPPORT = False
 
 bigquery_storage = _versions_helpers.BQ_STORAGE_VERSIONS.try_import()
 
@@ -572,9 +573,9 @@ def test_bq_to_arrow_array_w_conversion_fail(module_under_test):  # pragma: NO C
     series = pandas.Series(rows, name="test_col", dtype="object")
     bq_field = schema.SchemaField("field_name", "STRING", mode="REPEATED")
     exc_msg = f"""Error converting Pandas column with name: "{series.name}" and datatype: "{series.dtype}" to an appropriate pyarrow datatype: Array, ListArray, or StructArray"""
-    with pytest.raises(ArrowTypeError, match=exc_msg):
+    with pytest.raises(pyarrow.ArrowTypeError, match=exc_msg):
         module_under_test.bq_to_arrow_array(series, bq_field)
-        raise ArrowTypeError(exc_msg)
+        raise pyarrow.ArrowTypeError(exc_msg)
 
 
 @pytest.mark.parametrize("bq_type", ["RECORD", "record", "STRUCT", "struct"])
diff --git a/tests/unit/test__versions_helpers.py b/tests/unit/test__versions_helpers.py
index 8fa09962..b1d0ef1a 100644
--- a/tests/unit/test__versions_helpers.py
+++ b/tests/unit/test__versions_helpers.py
@@ -18,17 +18,17 @@ import pytest
 
 try:
     import pyarrow  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pyarrow = None
 
 try:
     from google.cloud import bigquery_storage  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     bigquery_storage = None
 
 try:
     import pandas  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     pandas = None
 
 from google.cloud.bigquery import _versions_helpers
@@ -39,11 +39,8 @@ from google.cloud.bigquery import exceptions
 def test_try_import_raises_no_error_w_recent_pyarrow():
     versions = _versions_helpers.PyarrowVersions()
     with mock.patch("pyarrow.__version__", new="5.0.0"):
-        try:
-            pyarrow = versions.try_import(raise_if_error=True)
-            assert pyarrow is not None
-        except exceptions.LegacyPyarrowError:  # pragma: NO COVER
-            raise ("Legacy error raised with a non-legacy dependency version.")
+        pyarrow = versions.try_import(raise_if_error=True)
+        assert pyarrow is not None
 
 
 @pytest.mark.skipif(pyarrow is None, reason="pyarrow is not installed")
@@ -62,6 +59,16 @@ def test_try_import_raises_error_w_legacy_pyarrow():
             versions.try_import(raise_if_error=True)
 
 
+@pytest.mark.skipif(
+    pyarrow is not None,
+    reason="pyarrow is installed, but this test needs it not to be",
+)
+def test_try_import_raises_error_w_no_pyarrow():
+    versions = _versions_helpers.PyarrowVersions()
+    with pytest.raises(exceptions.LegacyPyarrowError):
+        versions.try_import(raise_if_error=True)
+
+
 @pytest.mark.skipif(pyarrow is None, reason="pyarrow is not installed")
 def test_installed_pyarrow_version_returns_cached():
     versions = _versions_helpers.PyarrowVersions()
@@ -208,6 +215,16 @@ def test_try_import_raises_error_w_legacy_pandas():
             versions.try_import(raise_if_error=True)
 
 
+@pytest.mark.skipif(
+    pandas is not None,
+    reason="pandas is installed, but this test needs it not to be",
+)
+def test_try_import_raises_error_w_no_pandas():
+    versions = _versions_helpers.PandasVersions()
+    with pytest.raises(exceptions.LegacyPandasError):
+        versions.try_import(raise_if_error=True)
+
+
 @pytest.mark.skipif(pandas is None, reason="pandas is not installed")
 def test_installed_pandas_version_returns_cached():
     versions = _versions_helpers.PandasVersions()
diff --git a/tests/unit/test_legacy_types.py b/tests/unit/test_legacy_types.py
index 3431074f..809be185 100644
--- a/tests/unit/test_legacy_types.py
+++ b/tests/unit/test_legacy_types.py
@@ -19,7 +19,7 @@ import warnings
 
 try:
     import proto  # type: ignore
-except ImportError:  # pragma: NO COVER
+except ImportError:
     proto = None
 
 
diff --git a/tests/unit/test_opentelemetry_tracing.py b/tests/unit/test_opentelemetry_tracing.py
index e96e18c6..579d7b1b 100644
--- a/tests/unit/test_opentelemetry_tracing.py
+++ b/tests/unit/test_opentelemetry_tracing.py
@@ -19,7 +19,7 @@ from unittest import mock
 
 try:
     import opentelemetry
-except ImportError:  # pragma: NO COVER
+except ImportError:
     opentelemetry = None
 
 if opentelemetry is not None:
diff --git a/tests/unit/test_table.py b/tests/unit/test_table.py
index dbc5948b..3953170f 100644
--- a/tests/unit/test_table.py
+++ b/tests/unit/test_table.py
@@ -3408,6 +3408,7 @@ class TestRowIterator(unittest.TestCase):
 
     def test_to_dataframe_progress_bar(self):
         pytest.importorskip("pandas")
+        pytest.importorskip("pyarrow")
         pytest.importorskip("tqdm")
 
         from google.cloud.bigquery.schema import SchemaField
@@ -3427,7 +3428,6 @@ class TestRowIterator(unittest.TestCase):
 
         progress_bars = (
             ("tqdm", mock.patch("tqdm.tqdm")),
-            ("tqdm_notebook", mock.patch("tqdm.notebook.tqdm")),
             ("tqdm_gui", mock.patch("tqdm.tqdm_gui")),
         )
 
@@ -3444,9 +3444,43 @@ class TestRowIterator(unittest.TestCase):
             progress_bar_mock().close.assert_called_once()
             self.assertEqual(len(df), 4)
 
+    def test_to_dataframe_progress_bar_notebook(self):
+        pytest.importorskip("pandas")
+        pytest.importorskip("pyarrow")
+        pytest.importorskip("tqdm")
+        pytest.importorskip("tqdm.notebook")
+
+        from google.cloud.bigquery.schema import SchemaField
+
+        schema = [
+            SchemaField("name", "STRING", mode="REQUIRED"),
+            SchemaField("age", "INTEGER", mode="REQUIRED"),
+        ]
+        rows = [
+            {"f": [{"v": "Phred Phlyntstone"}, {"v": "32"}]},
+            {"f": [{"v": "Bharney Rhubble"}, {"v": "33"}]},
+            {"f": [{"v": "Wylma Phlyntstone"}, {"v": "29"}]},
+            {"f": [{"v": "Bhettye Rhubble"}, {"v": "27"}]},
+        ]
+        path = "/foo"
+        api_request = mock.Mock(return_value={"rows": rows})
+
+        with mock.patch("tqdm.notebook.tqdm") as progress_bar_mock:
+            row_iterator = self._make_one(_mock_client(), api_request, path, schema)
+            df = row_iterator.to_dataframe(
+                progress_bar_type="tqdm_notebook",
+                create_bqstorage_client=False,
+            )
+
+            progress_bar_mock.assert_called()
+            progress_bar_mock().update.assert_called()
+            progress_bar_mock().close.assert_called_once()
+            self.assertEqual(len(df), 4)
+
     @mock.patch("google.cloud.bigquery._tqdm_helpers.tqdm", new=None)
     def test_to_dataframe_no_tqdm_no_progress_bar(self):
         pytest.importorskip("pandas")
+        pytest.importorskip("pyarrow")
         from google.cloud.bigquery.schema import SchemaField
 
         schema = [
@@ -3711,7 +3745,7 @@ class TestRowIterator(unittest.TestCase):
         if hasattr(pandas, "Float64Dtype"):
             self.assertEqual(list(df.miles), [1.77, 6.66, 2.0])
             self.assertEqual(df.miles.dtype.name, "Float64")
-        else:  # pragma: NO COVER
+        else:
             self.assertEqual(list(df.miles), ["1.77", "6.66", "2.0"])
             self.assertEqual(df.miles.dtype.name, "string")
 
diff --git a/tests/unit/test_table_pandas.py b/tests/unit/test_table_pandas.py
index b3856856..02a7a6a7 100644
--- a/tests/unit/test_table_pandas.py
+++ b/tests/unit/test_table_pandas.py
@@ -16,11 +16,6 @@ import datetime
 import decimal
 from unittest import mock
 
-try:
-    import importlib.metadata as metadata
-except ImportError:
-    import importlib_metadata as metadata
-
 import pytest
 
 from google.cloud import bigquery
@@ -31,11 +26,6 @@ pyarrow = pytest.importorskip("pyarrow", minversion="3.0.0")
 
 TEST_PATH = "/v1/project/test-proj/dataset/test-dset/table/test-tbl/data"
 
-if pandas is not None:  # pragma: NO COVER
-    PANDAS_INSTALLED_VERSION = metadata.version("pandas")
-else:  # pragma: NO COVER
-    PANDAS_INSTALLED_VERSION = "0.0.0"
-
 
 @pytest.fixture
 def class_under_test():
@@ -44,7 +34,10 @@ def class_under_test():
     return RowIterator
 
 
-@pytest.mark.skipif(PANDAS_INSTALLED_VERSION[0:2] not in ["0.", "1."], reason="")
+@pytest.mark.skipif(
+    pandas.__version__.startswith("2."),
+    reason="pandas 2.0 changes some default dtypes and we haven't update the test to account for those",
+)
 def test_to_dataframe_nullable_scalars(monkeypatch, class_under_test):
     # See tests/system/test_arrow.py for the actual types we get from the API.
     arrow_schema = pyarrow.schema(