Camera2: Remove settings preferences only once

Browsing through camera settings crashes on devices with only
a back- or a front facing camera. Fix NPE on removing preferences
that are already removed.

Change-Id: I00b4549e18ea44975bf795da9384892d8ef2ee16