separate config into file and add botots.txt for api site