Access permissions can be limited for specific folders or files on an FTP server hosted on Google Drive by adjusting the sharing settings of those items. This allows for greater control over who can view and edit sensitive information.