With our previous discussion about shipping configs packaged in the jar vs as a part of the topology conf and then populating a Hadoop Configuration.  This is one of those places where some of the config really has to be on the class path, and cannot be a part of the topology conf.  UserGroupInformation internally is creating a new Configuration that will only search the class path and pull in configs it finds.  It then uses that config to decide if security is enabled.  If you don't have a core-site.xml that is configured with security, it will never think you have security turned on.