MediaSet: Do not erase element from children map before unref its children

Change-Id: Ic06878429859fd12855beb9fd4625ff6a874eb2a