Fix handling of .btn-group placed inside .input-group-btn

fixes #11909