If you're calling form fields directly in a layout instead of recursing over a form or fieldset using JForm's API, you don't need fields to be contained in a fieldset.  It's a good practice to follow, but not one that is strictly necessary.  [com_categories edit](https://github.com/joomla/joomla-cms/blob/master/administrator/components/com_categories/models/forms/category.xml) works just fine without fieldsets.