Currently, you can only make custom attributes & custom options active or inactive, you cannot delete them.
On the Settings / Custom Attributes list, you can toggle whether the custom attribute should be active (displayed on the custom attributes widget) or not.
Note that by default the Settings page is set to only display Active custom attributes, so if you want to activate a disabled attribute you will need to change the filter.
For Select attribute type, you can use the same mechanism on Options via the Active toggle.