Remove some unnecessary CONFIG_FOO_COMPONENT ifdefs

The files are only ever compiled if that condition is true.