When you use the phrase, ‘to me’, we usually use it to indicate when you want something or someone passed in your direction.  ‘Pass the flowers to me’ ‘At me’ is used when you want to express something was thrown in your direction or when someone’s actions were directed at you. ‘She threw the ball at me’ ‘He waved at me as soon as he entered the room’