Issue 1:
Code Sample, a copy-pastable example if possible

d=pd.DataFrame(
[
[1,2,3],
[1,2,4],
[2,3,5],
[1,None,5],
],
columns=['i','j','k']
)
print d.dtypes
d = d.set_index(['i','j'])['k']
#d = d.sort_index(level=[0,1])
d = d.sort_index()
print d.loc[(2,3)] # PerformanceWarning even though sorted

Problem description

This behavior his is very confusing. The docs ask me to sort multiindexes to avoid the performance warning,

If I do it, I still get the PerformanceWarning. If I print the dataframe, I see that data was indeed sorted.

This is resolved by explicitly specifying the levels to sort: d.sort_index(level=[0,1]).

As a user, I'm confused. Wasn't d.sort_index() the same thing as d.sort_index(level=...all levels...)? Inspecting my dataframe, it appeared to be the, case (the index was sorted, and nested into a hierarchy).

Shouldn't d.sort_index() simply be the same as d.sort_index(level=...all levels...)

I'm surprised there's any behavioral difference to be?

It seems due to:

d = d.sort_index()
print d.index.is_lexsorted() # False

d = d.sort_index(level=[0,1])
print d.index.is_lexsorted() # True

and then, they are only different if the index contains nulls.

Why there's a difference doesn't seem to be documented.

Expected Output

Either: it should be that d.sort_index() === d.sort_index(level=...all levels...)

or, dataframe.sort_index documentation could stand improvement and explain when the fully specified level parameter is different to not specifying level.

I'd prefer: d.sort_index() === d.sort_index(level=d.index.names), because:

this would allow functions function working on dataframes with vanilla indexes and calling sort_index(), also work optimally when provided dataframes with multindexes.

its cumbersome and error-prone to need to remember to specify the level names, since this is only needed in the multi-index case, and then also only in the case where the index contains NULLs.

Output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Linux
OS-release: 3.13.0-24-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.22.0

pytest: None

pip: 9.0.1

setuptools: 36.0.1

Cython: 0.27.3

numpy: 1.14.0

scipy: 1.0.0

pyarrow: None

xarray: None

IPython: 5.3.0

sphinx: None

patsy: 0.2.1

dateutil: 2.6.1

pytz: 2017.2

blosc: None

bottleneck: None

tables: 3.1.1

numexpr: 2.6.2

feather: None

matplotlib: 2.0.2

openpyxl: 1.7.0

xlrd: 0.9.2

xlwt: 0.7.5

xlsxwriter: None

lxml: None

bs4: None

html5lib: 0.999999999

sqlalchemy: 1.0.15

pymysql: None

psycopg2: 2.7.3.2 (dt dec pq3 ext lo64)

jinja2: 2.9.6

s3fs: None

fastparquet: None

pandas_gbq: None

pandas_datareader: None

Issue 2:
Code Sample, a copy-pastable example if possible

In [1]: import pandas as pd
In [2]: import numpy as np
In [3]: dates_range = pd.date_range('2016/10/1', periods=3).repeat(4)
In [4]: statuses = np.tile(['A', 'B', None, 'C'], 3)
In [5]: values = np.random.randn(12)
In [6]: df = pd.DataFrame({'date': dates_range, 'status': statuses, 'value': values})
In [7]: df
Out[7]:
date status value
0 2016-10-01 A -1.946876
1 2016-10-01 B 1.080243
2 2016-10-01 None 0.165715
3 2016-10-01 C -0.615913
4 2016-10-02 A 0.662645
5 2016-10-02 B 1.448593
6 2016-10-02 None -1.392233
7 2016-10-02 C 1.534083
8 2016-10-03 A 0.801988
9 2016-10-03 B -0.689987
10 2016-10-03 None -0.150036
11 2016-10-03 C -0.197410

In [8]: df.set_index(['date', 'status']).sort_index()
Out[8]:
value
date status
2016-10-01 A -1.946876
B 1.080243
C -0.615913
NaN 0.165715
2016-10-02 A 0.662645
B 1.448593
C 1.534083
NaN -1.392233
2016-10-03 A 0.801988
B -0.689987
C -0.197410
NaN -0.150036

In [9]: df.set_index(['date', 'status']).sort_index().index.is_lexsorted()
Out[9]: False

In [10]: df.set_index(['date', 'status']).sort_index(level=['date', 'status']).index.is_lexsorted()
Out[10]: True

Problem description

By giving sort_index() no level argument I expect to have the dataframe sorted on all the levels, as indeed it happens. However the is_lexsorted() method afterwards reports that it is not. The problem is fixed if I explicitly pass the level argument.

The behavior is particularly problematic when the MultiIndex has many levels, say n, and the NaNs appear at level m < n. Then slicing the MultiIndex for levels up to m will work, while for levels from m and above it will crash.

Note that this was not so in previous versions. Unfortunately not sure which ones; it worked OK in spring 2017, I came upon the issue by re-running code from that time.

Expected Output

The expected behavior is the one that I would get with passing the level argument with all index levels, as in the transcript above.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None

python: 3.6.3.final.0

python-bits: 64

OS: Darwin

OS-release: 17.0.0

machine: x86_64

processor: i386

byteorder: little

LC_ALL: en_US.UTF-8

LANG: en_US.UTF-8

LOCALE: en_US.UTF-8

pandas: 0.20.3

pytest: None

pip: 9.0.1

setuptools: 36.5.0

Cython: None

numpy: 1.13.3

scipy: 0.19.1

xarray: None

IPython: 6.2.0

sphinx: None

patsy: 0.4.1

dateutil: 2.6.1

pytz: 2017.2

blosc: None

bottleneck: None

tables: None

numexpr: None

feather: None

matplotlib: 2.0.2

openpyxl: None

xlrd: 1.0.0

xlwt: None

xlsxwriter: None

lxml: None

bs4: 4.6.0

html5lib: 0.999999999

sqlalchemy: 1.1.14

pymysql: 0.7.11.None

psycopg2: None

jinja2: 2.9.6

s3fs: None

pandas_gbq: None

pandas_datareader: None