Issue 1:
Code Sample, a copy-pastable example if possible

In [2]: df = pd.DataFrame([[1,2,5,6], [3,4,7,8]], index=['a', 'a'], columns=pd.MultiIndex.from_product([[1,2], ['A', 'B']]))

In [3]: df.loc['c'] = -1

In [4]: df.loc['c', (1, 'A')] = 3

In [5]: df.loc['d', (1, 'A')] = 3
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-5-34c3deb1645d> in <module>()
----> 1 df.loc['d', (1, 'A')] = 3

/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in __setitem__(self, key, value)
176 key = com._apply_if_callable(key, self.obj)
177 indexer = self._get_setitem_indexer(key)
--> 178 self._setitem_with_indexer(indexer, value)
179
180 def _has_valid_type(self, k, axis):

/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in _setitem_with_indexer(self, indexer, value)
348 index = self.obj._get_axis(i)
349 labels = index.insert(len(index), key)
--> 350 self.obj._data = self.obj.reindex_axis(labels, i)._data
351 self.obj._maybe_update_cacher(clear=True)
352 self.obj.is_copy = None

/home/pietro/nobackup/repo/pandas/pandas/core/frame.py in reindex_axis(self, labels, axis, method, level, copy, limit, fill_value)
2844 self).reindex_axis(labels=labels, axis=axis,
2845 method=method, level=level, copy=copy,
-> 2846 limit=limit, fill_value=fill_value)
2847
2848 @Appender(_shared_docs['rename'] % _shared_doc_kwargs)

/home/pietro/nobackup/repo/pandas/pandas/core/generic.py in reindex_axis(self, labels, axis, method, level, copy, limit, fill_value)
2492 limit=limit)
2493 return self._reindex_with_indexers({axis: [new_index, indexer]},
-> 2494 fill_value=fill_value, copy=copy)
2495
2496 def _reindex_with_indexers(self, reindexers, fill_value=np.nan, copy=False,

/home/pietro/nobackup/repo/pandas/pandas/core/generic.py in _reindex_with_indexers(self, reindexers, fill_value, copy, allow_dups)
2515 fill_value=fill_value,
2516 allow_dups=allow_dups,
-> 2517 copy=copy)
2518
2519 if copy and new_data is self._data:

/home/pietro/nobackup/repo/pandas/pandas/core/internals.py in reindex_indexer(self, new_axis, indexer, axis, fill_value, allow_dups, copy)
3879 # some axes don't allow reindexing with dups
3880 if not allow_dups:
-> 3881 self.axes[axis]._can_reindex(indexer)
3882
3883 if axis >= self.ndim:

/home/pietro/nobackup/repo/pandas/pandas/indexes/base.py in _can_reindex(self, indexer)
2733 # trying to reindex on an axis with duplicates
2734 if not self.is_unique and len(indexer):
-> 2735 raise ValueError("cannot reindex from a duplicate axis")
2736
2737 def reindex(self, target, method=None, level=None, limit=None,

ValueError: cannot reindex from a duplicate axis

Problem description

I think the reindexing code path shouldn't be taken at all (in this case - I'm not referring to the general design choice): anyway, the operation should work, coherently with the case where the index is unique and with the case where the row exists (In [4]).

Expected Output

Like In [4].

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None

python: 3.5.3.final.0

python-bits: 64

OS: Linux

OS-release: 4.7.0-1-amd64

machine: x86_64

processor:

byteorder: little

LC_ALL: None

LANG: it_IT.utf8

LOCALE: it_IT.UTF-8

pandas: 0.19.0+783.gcd35d22a0

pytest: 3.0.6

pip: 9.0.1

setuptools: 33.1.1

Cython: 0.25.2

numpy: 1.12.0

scipy: 0.18.1

xarray: 0.9.1

IPython: 5.1.0.dev

sphinx: 1.4.9

patsy: 0.3.0-dev

dateutil: 2.5.3

pytz: 2016.7

blosc: None

bottleneck: 1.2.0

tables: 3.3.0

numexpr: 2.6.1

feather: 0.3.1

matplotlib: 2.0.0

openpyxl: 2.3.0

xlrd: 1.0.0

xlwt: 1.1.2

xlsxwriter: 0.9.6

lxml: 3.7.1

bs4: 4.5.3

html5lib: 0.999999999

sqlalchemy: 1.0.15

pymysql: None

psycopg2: None

jinja2: 2.8

s3fs: None

pandas_gbq: None

pandas_datareader: 0.2.1

Issue 2:
I'm not sure if this functionality exists but it is boiler plate which I need to use quite often.

Typically I need to work with very large datasets, e.g. 10mil rows or more.

On these datasets performing a calculations such as taking a date in a specific string format can be sped up incredibly by assessing what needs to be done first.

As such I find myself writing the following boiler plate code for a lot of functions.

def map_strptime(series, format=None):
mapping = {x: datetime.strptime(x, format) for x in set(series)}
return series.map(mapping)

Now comparing this implementation v.s a more naive implementation just using the standard apply function, e.g.

def nonmap_strptime(x, format=None):
return datetime.strptime(x, format)

Usage:

df["Date"] = map_strptime(df["Trading Date"], format="%Y-%m-%d")

# vs.

df["Date"] = df["Trading Date"].apply(nonmap_strptime, format="%Y-%m-%d")

Assessing these two implementations for performance I get the following.

Note, my Dataset has 10 million rows across 365 days, e.g. a lot of repeated calculations.

# Mapped version
>>>> %timeit df["Date"] = map_strptime(df["Trading Date"], format="%Y-%m-%d")
>>>> 900 ms

# Nonmapped version
>>>> %timeit df["Date"] = df["Trading Date"].apply(nonmap_strptime, format="%Y-%m-%d")
>>>> (Still hasn't finished in the time it has taken me to write this issue, 10min plus)

As you can see for some use cases, e.g. when a calculation is being repeated often a simple set - map type wrapper around the operation could improve the performance substantially.

However, if a large number of non-duplicates exists then this would obviously slow the operation down quite substantially as first the calculation would be performed, then the mapping in two steps rather than one.

To get around this the number of unique items and non-unique items could be computed. If a large difference between them exists then a shift to the mapping implementation could impart large performance gains.

I'm not sure as to the best implementation, but something along the lines of:

df.memoapply(*args, **kargs):

if len(set(df[cols]) >> len(df[cols]):
# use memo version

else:
# Use standard versions.

This works best for one to one type operations, I'm unsure how well it would work for one to many or for many to one type applications.

Could possible be extended by using the inbuilt merge operations of Pandas and using a multi indexed series for the result?