Repository: pandas-dev/pandas
Pull request: PERF: asv for import
- [x] closes #26663

for py37 this uses `-X importtime` to get a more precise number without subprocess overhead.  Doesn't actually do anything with it, but what the heck.

Couple of unrelated cleanups.

Problem statement:
PERF: asv for import time
Not entirely sure how to do it with asv, but at the command line its just `python3 -X importtime -c "import pandas"`

- [ ] import time with `python3 -X importtime -c "import pandas"`
- [ ] import size with something like `import pandas, psutil; psutil.Process().memory_info()`

Hints from issue discussion:
Yea I think this would be a good idea. Maybe create a subprocess in setup and load pandas in the subprocess call
https://github.com/airspeed-velocity/asv/pull/487
Here is something similar numpy did: https://github.com/numpy/numpy/pull/8159/files - do we just want to benchmark `import pandas` or test some sub packages too? 
Makes sense to me. I think we'd take a PR for something like that if you'd like to post one
Ideally, the setup would ensure that matplotlib and numexpr  (really, all our optional deps) aren’t in the Python env running the benchmark. 

> On Jun 8, 2019, at 15:50, William Ayd <notifications@github.com> wrote:
> 
> Makes sense to me. I think we'd take a PR for something like that if you'd like to post one
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub, or mute the thread.

No opinion on numexpr, but for matplotlib I'd rather it be installed.  I'm pretty sure some effort has been put in to making matplotlib imports lazy.  This would help ensure that we don't accidentally undo that (this was motivated by exactly this happening in statsmodels)
Matplotlib has to be imported by default right now because we register formatters. That’s deprecated, and then we can ensure it’s lazily imported. 

We have a separate CI check ensuring that certain packages aren’t imported with pandas (search for import time)

> On Jun 9, 2019, at 10:33, jbrockmendel <notifications@github.com> wrote:
> 
> No opinion on numexpr, but for matplotlib I'd rather it be installed. I'm pretty sure some effort has been put in to making matplotlib imports lazy. This would help ensure that we don't accidentally undo that (this was motivated by exactly this happening in statsmodels)
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.

https://github.com/pandas-dev/pandas/blob/c7748ca057cad4a80a6c4cbf8b49c5d9f3e7ea1c/ci/code_checks.sh#L192
Thanks for pointing that out.  Is there a particular reason that was used instead of the approach here https://github.com/statsmodels/statsmodels/pull/5831 ?
You mean a separate check, not run as part of the pytest test suite? Probably no reason; seems easier to me that it be separate since it isn’t really a traditional unit test. 

> On Jun 9, 2019, at 11:43, jbrockmendel <notifications@github.com> wrote:
> 
> Thanks for pointing that out. Is there a particular reason that was used instead of the approach here statsmodels/statsmodels#5831 ?
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.

Commit message: missing whitespce

Patch to review:
diff --git a/asv_bench/benchmarks/package.py b/asv_bench/benchmarks/package.py
new file mode 100644
index 0000000000000..bd9752536f580
--- /dev/null
+++ b/asv_bench/benchmarks/package.py
@@ -0,0 +1,24 @@
+"""
+Benchmarks for pandas at the package-level.
+"""
+import subprocess
+import sys
+
+from pandas.compat import PY37
+
+
+class TimeImport:
+    def time_import(self):
+        if PY37:
+            cmd = [sys.executable, "-X", "importtime", "-c", "import pandas as pd"]
+            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            _, stderr = p.communicate()
+
+            line = stderr.splitlines()[-1]
+            field = line.split(b"|")[-2].strip()
+            total = int(field)  # microseconds
+            return total
+
+        cmd = [sys.executable, "-c", "import pandas as pd"]
+        p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        p.communicate()
diff --git a/pandas/core/dtypes/dtypes.py b/pandas/core/dtypes/dtypes.py
index ee1866e60644b..aa7e6801ba431 100644
--- a/pandas/core/dtypes/dtypes.py
+++ b/pandas/core/dtypes/dtypes.py
@@ -23,7 +23,7 @@
 ordered_sentinel = object()  # type: object
 
 
-def register_extension_dtype(cls: Type[ExtensionDtype],) -> Type[ExtensionDtype]:
+def register_extension_dtype(cls: Type[ExtensionDtype]) -> Type[ExtensionDtype]:
     """
     Register an ExtensionType with pandas as class decorator.
 
diff --git a/pandas/util/_test_decorators.py b/pandas/util/_test_decorators.py
index 627757aaa3741..0e07b9f5fe9f7 100644
--- a/pandas/util/_test_decorators.py
+++ b/pandas/util/_test_decorators.py
@@ -102,7 +102,7 @@ def _skip_if_no_scipy():
     )
 
 
-def skip_if_installed(package: str,) -> Callable:
+def skip_if_installed(package: str) -> Callable:
     """
     Skip a test if a package is installed.
 


Code review comments:
diff_hunk: @@ -0,0 +1,24 @@
+"""
+Benchmarks for pandas at the package-level.
+"""
+import subprocess
+import sys
+
+from pandas.compat import PY37
+
+
+class TimeImport:
+    def time_import(self):
+        if PY37:
+            cmd = [sys.executable, "-X", "importtime", "-c", "import pandas as pd"]
path: asv_bench/benchmarks/package.py
text: @user1:
is there a reason you are not using ``subprocess.run(...).stdout`` ? much simpler

@author:
force of habit, will change

@author:
updated