Adding a warning when dropping NA values for panel.to_frame #7879
State: closed

closes: https://github.com/pydata/pandas/issues/7879

I've added a warning when the <code>NaN</code> values are dropped when calling <code>pd.Panel(dict).to_frame()</code> with the param <code>filter_observations</code> true

I've supplied a test case to be used against the example provided in the issue tracker


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: API Design

Comments:
Thanks for the PR.

I added a couple comments. Let me know if they don't make sense or you need any help.

their is another pr for this issue

however I think in that one the consensus was to make an API change to make filter_observations to False

instead of doing a warning which I think is a bit odd
as we don't warn on anything else like this

@TomAugspurger, I've modified the test as following

```
def test_to_frame_na_drop_warnings(self):
        df1 = DataFrame(np.random.randn(2, 3), columns=['A', 'B', 'C'],
               index=['foo', 'bar'])
        df2 = DataFrame(np.random.randn(2, 3), columns=['A', 'B', 'C'],
               index=['foo', 'bar'])
        df2.loc['foo', 'B'] = np.nan
        dict_without_dropped_vals = {'df1': df1, 'df2': df2}
        ## A panel without dropped vals shouldn't throw warnings
        with tm.assert_produces_warning(False):
            Panel(dict_without_dropped_vals).to_frame()
        ## A panel with dropped vals should throw a Runtime warning if \
        # filter_observations is True
        df2_with_na_vals = DataFrame(df2)
        df2_with_na_vals.loc['foo', 'B'] = np.nan
        dict_with_dropped_vals = {'df1': df1, 'df2_dropped': df2_with_na_vals}
        with tm.assert_produces_warning(RuntimeWarning):
            Panel(dict_with_dropped_vals).to_frame()
        ##if filter_observations is False, a warning shouldn't be throws
        with tm.assert_produces_warning(False):
            Panel(dict_with_dropped_vals).to_frame(filter_observations=False)
```

Does this match what you expected?

Thanks for the reply

@jreback So you suggest I stop working on that PR as pandas doesn't use warnings by default, Right?
Concerning the other PR, was it accepted, I didn't find any PRs or comments referencing it?
Thanks

see #7970. Its not been accepted as of yet.

pandas _does_ use warnings. Its for this usecase.

I _think_ the API change is better. Really just looking for some feedback (and then impl of course).

@Magellanea pls review that PR and post comments (here is ok).

@jreback  Concerning the API change - if I'm we're going to neglect the warnings - do you suggest that we evaluate `filter_observation` to be false regardless of the passed value, Is that what you mean by the API CHANGE, Thanks

@Magellanea no I meant change the default of `filter_observations=True` to `False`. I think this would be fine as an API change with an example in v0.15.0.txt. You will have a couple of test fialures I think which will need addressing.

@Magellanea can you rebase / address comments?

@jreback I'm sorry for leaving this for a while, the problem was that the travis built failed, and I don't know what the reason was, I've changed the `filter_observation` default value and in the corresponding test but the built failed for other reasons and irrelevant tests

dupe of #7970