Passing him back and forth, bouncing him, repeatedly trying to feed him, etc. will only get him more worked up. If you feel certain that your baby is not hungry, dirty, or in pain, then there is nothing wrong with putting him down in his crib, where you know he will be safe, and walking away. Close the door, turn on some quiet music, and relax for a few minutes. If he is still crying after 15 to 20 minutes, go back in and wrap him up snugly, and hold him again. Sometimes, after letting off some steam this way, babies will go right to sleep.
Some doctors like to start vitamin drops, depending on particular circumstances.
Ask your doctor about this if you have any questions. Other than this, your baby does not need anything but your breast milk. It is possible to over-feed a baby if you stick a bottle in his mouth every time he cries, but you usually have to work at it. As long as you look for other ways to calm him when he is not really hungry, then you should do just fine. Please ask your health care provider about any questions you may have.