PERF: faster placement creating extension blocks from arrays
State: closed

When creating a DataFrame from many arrays stored in ExtensionBlocks, it seems quite some time is taken inside BlockPlacement using `np.require` on the passed list. Specifying the placement as a slice instead gives a much faster creation of the BlockPlacement. This delays the conversion to an array, though, but afterwards the conversion of the slice to an array inside BlockPlacement when neeeded is faster than an initial creation of a BlockPlacement from a list/array of 1 element.

From investigating https://github.com/pandas-dev/pandas/issues/32196#issuecomment-600824238

@rth this reduces it with another third! (only from the dataframe creation, to be clear)

Labels:
name: Performance
description: Memory or execution speed performance
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Using the same example from https://github.com/pandas-dev/pandas/pull/32826. With:

```
arrays = [pd.arrays.SparseArray(np.random.randint(0, 2, 1000), dtype="float64") for _ in range(10000)]
index = pd.Index(range(len(arrays[0])))  
columns = pd.Index(range(len(arrays)))    
```

it gives

```
In [4]: %timeit pd.DataFrame._from_arrays(arrays, index=index, columns=columns)  
113 ms ± 874 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)   <-- master
72.9 ms ± 648 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)   <-- PR
```
This is also indirectly covered by the sparse benchmark, but adding some benchmarks specifically for `_from_arrays`:

```
       before           after         ratio
     [e31354f0]       [6d6e822a]
     <master>         <perf-placement>
-      14.5±0.7ms       9.58±0.6ms     0.66  frame_ctor.FromArrays.time_frame_from_arrays_sparse
```
another option would be to pass BlockPlacement objects.  If we do that consistently, we could remove the checks/casting in the constructor/property
Sorry, I don't understand that option. You still need to create BlockPlacement objects, right? And the question here is about how to create them (from an integer, or from a slice, or a 1-len list)
> You still need to create BlockPlacement objects, right? And the question here is about how to create them (from an integer, or from a slice, or a 1-len list)

Yah, the thought was about creating the BlockPlacement object and passing it to `make_block` rather than having it created later.  Can ignore as orthogonal, but might trim some overhead.
> passing it to make_block rather than having it created later. 

It shouldn't matter much I think, it is just passed through until ExtensionBlock init, and there it's doing a `if not isinstance(placement, BlockPlacement): placement = BlockPlacement(placement)`

So we would first need to eliminate all other places where we pass a slice/array as placement to block creation, and then it would only elimiate one isinstance call
> So we would first need to eliminate all other places where we pass a slice/array 

Yes, it is not a trivial idea.

> and then it would only elimiate one isinstance call

We could also make mgr_locs not-a-property, so get marginally faster lookups.

But again, ignore as orthogonal.
Yep, any other comments on the PR itself?
LGTM
thanks @jorisvandenbossche