There is no doubt that you should contact a de addiction centre, a psychologist or at least have some sort of medical supervision while you attempt to give up the use of drugs. You must understand that even though the physical withdrawal symptoms may have subsided and you may be drug free, mentally there may still be withdrawals which are being manifested as fears and anxieties. There is no fixed time duration for how long these mental withdrawals can last and nor is it linked to any brain damage, which must be looked into separately. However, you must also look into the cause of your drug abuse and continue your abstinence along with an emphasis on confidence building.