Take things back a few steps and get the hamster used to your voice and being around him. Then wash your hand so it doesn't smell of food, and let him get used to your scent. Try stroking him and bit by bit win his confidence. When you first pick him up keep your hand low, so that he isn't whizzed up into the air which will be scary for him. Be patient as for fearful hamster it can take weeks or even months to win him over.