Make sure we remove the shard metadata whenever it expires