PERF: fastpath DataFrame constructor from BlockManager
State: closed

When trying to make `fast_apply` unnecessary I found that there is non-trivial overhead in the constructors that we can avoid.  Moreover, since users shouldn't be passing BlockManagers around anyway, we _might_ be able to get that case out of the DataFrame constructor entirely.

```
In [2]: ser = pd.Series(range(5))                                                                                                                                        
In [3]: df = ser.to_frame()                                                                                                                                              
In [4]: mgr = df._mgr

In [5]: %timeit pd.DataFrame(mgr)                                                                                                                                       
2.08 µs ± 51.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [6]: %timeit pd.DataFrame._from_mgr(mgr)                                                                                                                             
1.26 µs ± 17.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
```


Labels:
name: Performance
description: Memory or execution speed performance
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
> Moreover, since users shouldn't be passing BlockManagers around anyway, we _might_ be able to get that case out of the DataFrame constructor entirely.

makes sense but would changing obj._constructor -> type(obj)._from_mgr create issues with subclassing. I recall a recent discussion that _constructor should be type(self) 
> makes sense but would changing obj._constructor -> type(obj)._from_mgr create issues with subclassing.

Fair point.  If _constructor is something other than `type(self)` this would be change behavior.  Will update.

> I recall a recent discussion that _constructor should be type(self)

This sounds familiar.  I'd be +1.
>  Moreover, since users shouldn't be passing BlockManagers around anyway, we might be able to get that case out of the DataFrame constructor entirely.

This is however used by downstream libraries (eg pyarrow).

> I recall a recent discussion that _constructor should be type(self)

That is currently not required to be the case (and for geopandas it also is not the case). After that earlier discussion, this issue was opened about that: https://github.com/pandas-dev/pandas/issues/32638
@jorisvandenbossche thanks for the link. you wrote

> we also don't require _constructor to be a class, it can be a constructor function

so https://github.com/pandas-dev/pandas/pull/33357#discussion_r404992724 is probably not applicable.
The only requirement we have is that `_constructor` returns a callable, so `obj._constructor._from_mgr` will not necessarily work.
> Basically check that data is a BlockManager and that all other keywords are None? (that are some more checks of course, so would need to check if that doesn't undo the gain)

if this or something like it isnt accepted, then that may be worthwhile.  if we do get this fastpath, then id prefer not to make the already-complicated constructor more complicated.

> The only requirement we have is that _constructor returns a callable, so obj._constructor._from_mgr will not necessarily work.

How about

```
def _from_mgr(self, mgr):
    constr = self._constructor
    if constr is type(self):
         [the current implementation]
    else:
        return constr(mgr)
```

> How about

Then I personally think having the shortcut in the main constructor is the simpler solution
On further reflection, I'm on board with Joris's suggestion.  If at some point we add restrictions to _constructor that allow _from_mgr to be re-simplified, ill revisit at that point.
Thanks Brock!