Issue 1:
We've found a problem where repeating a row and a column in a DataFrame fails with a "Cannot create BlockManager._ref_locs" assertion error.

The dataframe is very simple:

df = pd.DataFrame(np.arange(25.).reshape(5,5),
index=['a', 'b', 'c', 'd', 'e'],
columns=['a', 'b', 'c', 'd', 'e'])

And we pull the data out like this:

z = df[['a', 'c', 'a']]
z.ix[['a', 'c', 'a']]
Traceback (most recent call last):
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/ipython-1.1.0_1_ahl1-py2.7.egg/IPython/core/interactiveshell.py", line 2830, in run_code
exec code_obj in self.user_global_ns, self.user_ns
File "<ipython-input-87-3bdc0aacc4b5>", line 1, in <module>
z.ix[['a', 'c', 'a']]
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/indexing.py", line 56, in __getitem__
return self._getitem_axis(key, axis=0)
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/indexing.py", line 744, in _getitem_axis
return self._getitem_iterable(key, axis=axis)
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/indexing.py", line 816, in _getitem_iterable
convert=False)
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/generic.py", line 1164, in take
new_data = self._data.take(indices, axis=baxis)
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/internals.py", line 3366, in take
ref_items=new_axes[0], axis=axis)
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/internals.py", line 2337, in apply
do_integrity_check=do_integrity_check)
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/internals.py", line 1990, in __init__
self._set_ref_locs(do_refs=True)
File "/users/is/dbew/pyenvs/timeseries/lib/python2.7/site-packages/pandas-0.13.0_292_g4dcecb0-py2.7-linux-x86_64.egg/pandas/core/internals.py", line 2130, in _set_ref_locs
'have _ref_locs set' % (block, labels))
AssertionError: Cannot create BlockManager._ref_locs because block [FloatBlock: [a], 1 x 3, dtype: float64] with duplicate items [Index([u'a', u'c', u'a'], dtype='object')] does not have _ref_locs set

If instead we take a copy of the intermediate step, then it works:

z = df[['a', 'c', 'a']].copy()
z.ix[['a', 'c', 'a']]
Out[89]:
a c a
a 0 2 0
c 10 12 10
a 0 2 0

[3 rows x 3 columns]

This means that if you several functions which each do a part of the data processing, you need to know the history of an object to know whether what you're doing works. I think .ix should always succeed on a DataFrame or Series, regardless of how it was constructed.

(I've read the discussion at #6056 about chained operations - but it's not something you can avoid if you have a pipeline of small steps instead of one big step).

This wasn't an issue in 0.11.0 but is failing in 0.13.0 and the latest master. Here's the output of installed versions when running on the master:

commit: None

python: 2.7.3.final.0

python-bits: 64

OS: Linux

OS-release: 2.6.18-308.el5

processor: x86_64

byteorder: little

LC_ALL: None

LANG: en_GB

pandas: 0.13.0-292-g4dcecb0

Cython: 0.16

numpy: 1.7.1

scipy: 0.9.0

statsmodels: None

patsy: None

scikits.timeseries: None

dateutil: 1.5

pytz: None

bottleneck: 0.6.0

tables: 2.3.1-1

numexpr: 2.0.1

matplotlib: 1.1.1

openpyxl: None

xlrd: 0.8.0

xlwt: None

xlsxwriter: None

sqlalchemy: None

lxml: 2.3.6

bs4: None

html5lib: None

bq: None

apiclient: None

Issue 2:
Code Sample, a copy-pastable example if possible

# Your code here
ParserErrorTraceback (most recent call last)
<ipython-input-9-4892e831abe8> in <module>()
1 import pandas as pd
----> 2 data = pd.read_csv("https://github.com/vaibhaw1994kumar/MachineLearning-test-cases/blob/master/international-airline-passengers.csv",usecols=[1],engine='python',skipfooter=3)

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, doublequote, delim_whitespace, low_memory, memory_map, float_precision)
676 skip_blank_lines=skip_blank_lines)
677
--> 678 return _read(filepath_or_buffer, kwds)
679
680 parser_f.__name__ = name

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in _read(filepath_or_buffer, kwds)
444
445 try:
--> 446 data = parser.read(nrows)
447 finally:
448 parser.close()

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in read(self, nrows)
1034 raise ValueError('skipfooter not supported for iteration')
1035
-> 1036 ret = self._engine.read(nrows)
1037
1038 # May alter columns / col_dict

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in read(self, rows)
2237 def read(self, rows=None):
2238 try:
-> 2239 content = self._get_lines(rows)
2240 except StopIteration:
2241 if self._first_chunk:

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in _get_lines(self, rows)
2969 while True:
2970 new_row = self._next_iter_line(
-> 2971 row_num=self.pos + rows + 1)
2972 rows += 1
2973

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in _next_iter_line(self, row_num)
2715 msg += '. ' + reason
2716
-> 2717 self._alert_malformed(msg, row_num)
2718 return None
2719

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in _alert_malformed(self, msg, row_num)
2674
2675 if self.error_bad_lines:
-> 2676 raise ParserError(msg)
2677 elif self.warn_bad_lines:
2678 base = 'Skipping line {row_num}: '.format(row_num=row_num)

ParserError: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?. Error could possibly be due to parsing errors in the skipped footer rows (the skipfooter keyword is only applied after Python's csv library has parsed all rows).

Problem description

[this should explain why the current behaviour is a problem and why the expected output is a better solution.]

Note: We receive a lot of issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates!

Note: Many problems can be resolved by simply upgrading pandas to the latest version. Before submitting, please check if that solution works for you. If possible, you may want to check if master addresses this issue, but that is not necessary.

For documentation-related issues, you can check the latest versions of the docs on master here:

https://pandas-docs.github.io/pandas-docs-travis/

If the issue has not been resolved there, go ahead and file it in the issue tracker.

Expected Output

data should be uploaded

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]