PERF: DataFrame(ndarr, copy=True)
State: closed

- [X] closes #52438
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Performance example:

```python
>>> import numpy
>>> import pandas as pd, numpy as np
>>> data = numpy.random.rand(10_000_000, 2)
>>> df = pd.DataFrame(data, copy=True)
>>> %timeit df.sum()
164 ms ± 1.12 ms per loop  # main
13.2 ms ± 48.3 µs per loop   # this PR
```



Labels:
name: Performance
description: Memory or execution speed performance
name: Mothballed
description: Temporarily-closed PR the author plans to return to
name: Copy / view semantics
name: Stale

Comments:
I've always had trouble using asv, IDK if'm doing it wrong? for example running `asv continuous -f 1.1 upstream/main HEAD -b groupby.GroupByMethods` takes 48 minutes on my macbook m1, which just isn't tenable for any serious benchmarking work.

 How long does `asv continuous -f 1.1 upstream/main HEAD -b groupby.GroupByMethods` take for you?
Not sure I've ever ran the groupby ones, the frame benchmarks for example take around 5-10 minutes (not totally sure, would lean more towards 5) on my m2 air
So `asv continuous -f 1.1 upstream/main HEAD -b frame_methods` takes approx 5 min. in for you?

Did you install asv like in the [docs](https://pandas.pydata.org/docs/development/contributing_codebase.html#running-the-performance-test-suite) i.e. (`pip install git+https://github.com/airspeed-velocity/asv`) or did you use conda or pip?

(I need to try figuring it out, so appreciate any help)
I think so yes, can re-run later today. It's definitely not terribly long

I used mamba (came when installing environment.yml)
5 was a bit optimistic, ran around 9ish minutes. 

2 suggestions:

- set the number of cores when compiling in asv.conf.json to the number of cores that your machine has (speeds it up quite nicely).
- check that you are not on low power mode (battery settings). test suite takes 2.5 times as long for me when on low power mode
Great, thanks I will try that, alternatively do a new install using mamba.
Ok, I've run asv tests, generally look good

```
asv continuous -f 1.1 upstream/main HEAD -b arithmetic

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
```

```
asv continuous -f 1.1 upstream/main HEAD -b frame_ctor

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
```

```
asv continuous -f 1.1 upstream/main HEAD -b stat_ops

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
```

```
asv continuous -f 1.1 upstream/main HEAD -b groupby.AggFunctionsx

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
```

```
asv continuous -f 1.1 upstream/main HEAD -b groupby

[100.00%] ··· groupby.MultiColumn.time_lambda_sum                                                                                                                                                  152±0.5ms
       before           after         ratio
     [c537b363]       [1160d3f6]
     <master>         <dataframe_2d_array_copy_true>
+      31.7±0.1μs         35.0±6μs     1.10  groupby.GroupByMethods.time_dtype_as_group('uint', 'cumsum', 'direct', 5)
-        310±80ms        246±0.6ms     0.79  groupby.AggEngine.time_dataframe_numba(True)
```

Everything else looks good so I guess it's jitters.

Generally, DataFrame construction with copy=True doesnøt seem to be suppported in the asv tests.   Below are the main differencers that I've noticed.

```python
>>> arr = np.random.randn(10_000_000, 2)
>>> %timeit df2 = pd.DataFrame(arr, copy=True)
13.5 ms ± 157 µs per loop  # main
21.5 ms ± 435 µs per loop  # this PR
>>> df.sum()
160 ms ± 469 µs per loop  # main
13.1 ms ± 57.9 µs per loop  # this PR
```

I.e. faster aggregation, but slower construction
Yeah sorry, running them agains main doesn't make much sense. You probably have to change the default to True and then run your change against that commit to get a better idea how things would change
Hmm, setting the default `copy=True`  and comparing against that will do a lot of other things also. In  particular, every time `DataFrame._constructor` & `Series._constructor_expanddim` are called, a copy will be made, which will be quite costly. That isn't proposed in this PR, so is not really the same.

This PR doesn't do anything that `DataFrame.copy` doesn't do already, so will logically have comparable performance to dataframes that come out of `DataFrame.copy()`. So I don't see any possibility that e.g. arithmetic operations can be affected negatively from this.

I could add a test that does `pd.DataFrame(data, copy=True/False).sum()` somewhere to in the ASVs to document the case where this gets a performance boost?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
I saw some very weird things when switching this for CoW. So I am not convinced that we should do this right now. I'd be more on-board with doing this in 3.0. I don't think that setting copy=True manually is a very common use-case anyway.
Guess there's not much appetite for this before 3.0 so will mothball this PR in the meantime