I will define a new enum for the return value. But for the name of this API, I should prefer CheckAPIAccessControl than CheckExtensionAccessControl. For the reason, let me introduce the background of this API firstly.  
1. Application's invocation of security sensitive API should be allowed or denied based on the permissions that the application has. These sensitive APIs are provided by extension. Because extension is shared by applications, caller should tell extension which application call it's API.
2. This API should be called by each exposed API of extension. This feature is for access control on API levels.
That's why I prefer CheckAPIAccessControl as the name.