You can use your elbow for testing. The water should be lukewarm. You might even put a little sponge or a towel at the bottom of the sink or tub so the baby doesn't slip. When you're holding the baby for a bath, reach around his back and under the opposite arm so you have a firm grip. Be sure to support the neck and head, because your baby doesn't have much head control for the first month or so.
Babies are a lot less fragile than they seem.
But you do have to be careful to never leave a baby unattended. Test the water with your elbow.